Until Futher notice this API is officialy down, until I can manage to access gptj or use a new model completely

A GPT-J API to use with python

pip install gptj

prompt: the prompt you wish to give to the model

tokens: the number of tokens to generate (values 204 or less are recommended)

temperature: controls the randomness of the model. higher values will be more random (suggestest to keep under 1.0 or less, something like 0.3 works)

top_p: top probability will use the most likely tokens

top_k: Top k probability

rep: The likely hood of the model repeating the same tokens lower values are more repetative

user: the speaker the person who is giving gpt-j a prompt

bot: an imaginary character of your choice

context: the part of the prompt that explains what is happening in the dialog

examples: a dictionary of user intentions and how the bot should respond

from gpt_j.Basic_api import simple_completion

prompt = "def perfect_square(num):"

max_length = 100

A low temperature means the model will take less changes when completing a prompt

A high temperature will make the model more creative

Both temperature and top probability must be a float

temperature = 0.09

If you are using top probability set temperature one

If you are using temperature set top probability to one

top_probability = 1.0

top_k = 40

repetition = 0.216

Here you set query equal to the desired values

Note values higher than 512 tend to take more time to generate

query = simple_completion(prompt, length=max_length, temp=temperature, top_p=top_probability, top_k=top_k, rep=repetition)

print(query)

from gpt_j.gptj_api import Completion

context = "This is a calculator bot that will answer basic math questions"

Queries are to the left while target responses should be to the right

Here we can see the user is asking the model math related questions

The way the model should respond if given on the right

DO NOT USE PERIODS AT THE END OF USER EXAMPLE!

examples = {
    "5 + 5": "10",
    "6 - 2": "4",
    "4 * 15": "60",
    "10 / 5": "2",
    "144 / 24": "6",
    "7 + 1": "8"}

context_setting = Completion(context, examples)

prompt = "48 / 6"

Below you can change student to "Task" for example and get similar results

User = "Student"

Bot = "Calculator"

max_tokens = 50

A low temperature means the model will take less changes when completing a prompt

A high temperature will make the model more creative and produce more random outputs

A Note both temperature and top probability must be a float

temperature = 0.09

If you are using it set temperature one

If you are using temperature set top probability to one

top_probability = 1.0

top_k = 40

repetition = 0.216

Unfilled parameters will be default values

I recommend all parameters are filled for better results

Once everything is done execute the code below

response = context_setting.completion(prompt,
              user=User,
              bot=Bot,
              max_tokens=max_tokens,
              temperature=temperature,
              top_p=top_probability,
              top_k=top_k,
              rep=reptition)

Please be patient depending on the given parameters it will take longer sometimes

For quick responses just use the Basic API which is a simplified version

print(response)

Note: This a very small model of 6B parameters and won't always produce accurate results

I have removed the security from the API, please don't use for unethical use! I am not responsible for anything you do with the API

This is all possible thanks to https://github.com/vicgalle/gpt-j-api

Feel free to check out the original API

© Michael D Arana

licensed under the MIT License.