Using GPT-3 to build Skill Data
Now that OpenAI has removed the wait list for GPT-3 I am keen to experiment what might be possible with this much talked about AI NLP model.
GTP-3 caught my attention when it was first released in 2020, its flexibility in performing a range of tasks with human-like accuracy is what makes it so exciting. And being a black-box API, developers like me can use to create AI-powered apps without extensive knowledge of deep-learning. In the past I have experimented with AWS services Rekognition and Personalize. Don’t be fooled, these services make AI more accessible but there’s still plenty to learn to use them effectively and GPT-3 is no exception.
Prompt Engineering
Prompt engineering is what drives the GPT-3 API. Because GPT-3 can do so many things, you need to be specific in what you want from it. The playground is a great place to experiment with prompt engineering.
From my initial testing, a good prompt is one that “shows” what the result should look like. GPT-3 responses well to instructions and examples. General terminology for prompt design:
Zero-shot learning: Where no examples are given for training.
One-shot learning: Here only one example is provided for the training purpose
Few-shot learning: Where a few examples are given to train the model along with the prompt.
Skills Data
Skills data is the measurement of what people in your organisation can do. I reckon most organisations struggle to maintain reliable skills data that is both valid and fresh. For those that have reliable skills data it’s not simply about accessing the data, it’s about extracting meaning from the data and using it to identify gaps and opportunities.
Can GPT-3 understand skills ?
Using Skills Data from Precise, I set about applying my GPT-3 knowledge to:
Creating a Skills Taxonomy: Can GPT-3 categorise and organise skills? If so, structured skills will help derive meaning from skill data
Creating a Skills Graph: Can GPT-3 understand relationships between skills? If so, it could help identify growth opportunities for people.
Precise is a SaaS product used by technology consultancies to create consultant profiles that contain a consultant’s experiences and skills.
Skills Taxonomy Prompt
Prompt:
A table summarizing the skills: Java, React, Hadoop, Hive, Pig, Python, Javascript, Node
| Technology | Category | Label | Usage | Application |Description:
Here we are asking GPT-3 to create a table for a set of given skills. The table has predefined headers. This is an example of zero-shot learning. That is the system is classifying the skills based on its own trained world knowledge. It has no prior examples to use as reference.
GTP-3 playground output:
Observation:
Overall the output if very good considering no prior examples were given. It has done a good job classifying the skills. The chosen attributes are accurate and consistent.
While the results with zero shot learning are pretty good, let's take a look at what we can do when we give GPT-3 some examples of what we want our output to be.
By providing GPT-3 a few examples to train its model allows it to create a deeper understanding of the prompt and start creating more meaningful output. Examples help steer the model in the right direction.
Few-shot learning example:
Here we have added two examples to the prompt. One for Java and Hadoop. We have also added new skills to the list Flutter, C++ and TensorFlow. The ### token is used to differentiate between different training examples.
GTP-3 playground output:
Observation:
Like the previous results, output is very good! Providing examples to model does help increase its accuracy. We are able to define a skill taxonomy in only a few examples that GPT-3 can understand. The taxonomy defined here is just for illustration, further work is required on this before advancing further.
Skills Graph Prompt
Let’s see if GPT-3 can understand relationships between skills. If so, this will be exciting. Skills become meaningful when they are connected to other things. A simple skill name is of limited meaning. Connecting skills to other skills gives them context and creates opportunities for meaningful insights — for both organisations and individuals.
Few-shot learning example:
Training the model with a few examples will be needed for this prompt. We give it three skills each with a list of related skills. The prompt tells GPT-3 to complete the related skills for “Ruby on Rails” - a populate framework for building web applications.
The same prompt was run with output from the previous. In this case:
skills related to Databases
skills related to SQL
Collecting the output as a graph:
Observation:
Again, results are impressive considering the test was thrown together in just a few minutes. Expanding the graph to include nodes other than skills should be consider. Other nodes may include behaviours, roles, outcomes. I am not expecting every node to be correct, but keep in mind, it was generated using a general purpose AI model with very little training with an overall investment of about 10 mins!
Summary
We’ve seen how a general purpose NPL model can produce some great results. It has performed well at creating a skills taxonomy and a skills graph with very little training. By creating more structured skills data, new possibilities can be explored. The skills graph can be used to develop future skills and find growth opportunities. The skills taxonomy can help people understand the skills they have.