dotAI REST API Resources

The dotAI feature offers a wealth of API tools to undertake OpenAI operations headlessly. Similar to the viewtool, the endpoints include those that focus on generation, searching, embeddings, and completions.

The various endpoints exist as child paths of /api/v1/ai, outlined below by category and path.

Generative Endpoints

/api/v1/ai/text/generate

When called with GET, only a prompt path parameter is required:

curl -X 'GET' \
  'http://localhost:8082/api/v1/ai/text/generate?prompt=vacationing%3F' \
  -H 'accept: application/json'

The response object resembles that of the viewtool's $ai.completions.raw() method:

{
  "id": "chatcmpl-8rREz6irGDpHEqRbOIGQSaq1MKb8n",
  "object": "chat.completion",
  "created": 1707746789,
  "model": "gpt-3.5-turbo-16k-0613",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Vacationing is a great way to relax, recharge, and explore new places. Whether you prefer a beach getaway, a mountain retreat, or a city adventure, there are endless possibilities for your vacation. From sunbathing on the beach, hiking in nature, visiting historical landmarks, or trying local cuisines, vacationing allows you to escape your daily routine and immerse yourself in new experiences. Whether you're planning a solo trip, a family vacation, or a romantic getaway, make sure to plan ahead, pack accordingly, and take the time to enjoy every moment of your well-deserved vacation."
      },
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 38,
    "completion_tokens": 121,
    "total_tokens": 159
  },
  "system_fingerprint": null
}

When called with POST, a data payload is instead required, of the type described under the section on completion forms, providing a greater range of configurability. However, the actual submissions can be as simple as only including a prompt property, as follows:

curl -X 'POST' \
  'http://localhost:8082/api/v1/ai/text/generate' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "prompt": "tell me about costa rica!"
}'

/api/v1/ai/image/generate

When called with GET, only a prompt path parameter is required:

curl -X 'GET' \
  'http://localhost:8082/api/v1/ai/image/generate?prompt=a%20pig%20wearing%20a%20hat' \
  -H 'accept: application/json'

The result is as follows:

{
  "revised_prompt": "A playful farm scene showcasing a pink pig. This pig is unique, it is wearing a hat possibly a sunhat for the bright sunny day. The hat is delicately perched on the pig's big head. Attention is drawn to the pig's gleeful expressions, and the curious sparkle in its eyes. Different shades of pink colour are reflected on its body indicating a healthy pig with a tinge of fun and eccentricity with the presence of the hat. The whole ambiance of the farm is of warmth and vitality. A light breeze may be rustling through making the image lively.",
  "url": "https://oaidalleapiprodscus.blob.core.windows.net/private/org-qZnNeZvyp7uFUh2EX8AJA6gw/user-2MAFKcfsfuJ7ILKoM3ZYWUcs/img-sLsWAgFW0B7U5cLaRblzmQeF.png?st=2024-02-12T13%3A29%3A31Z&se=2024-02-12T15%3A29%3A31Z&sp=r&sv=2021-08-06&sr=b&rscd=inline&rsct=image/png&skoid=6aaadede-4fb3-4698-a8f6-684d7786b067&sktid=a48cca56-e6da-484e-a814-9c849652bcb3&skt=2024-02-12T14%3A27%3A13Z&ske=2024-02-13T14%3A27%3A13Z&sks=b&skv=2021-08-06&sig=qwmBl4uETTfn14v4jQhUUUeH1HihtKm2m4udyxlmm%2BE%3D",
  "originalPrompt": "a pig wearing a hat",
  "tempFileName": "pig_20240212_022931.png",
  "response": "temp_298431dce8"
}

Calling this path with the POST method instead requires a JSON data payload with three properties: a prompt string, a numberOfImages integer, and a size string corresponding to one of the selectable image dimensions: 256x256, 512x512, 1024x1024,1024x1792, or 1792x1024.

Example:

curl -X 'POST' \
  'http://localhost:8082/api/v1/ai/image/generate' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "prompt": "a ball of string",
  "numberOfImages": 1,
  "size": "1024x1024"
}'

Search Endpoints

/api/v1/ai/search

When performing a GET call to ai/search, a list of query parameters are accepted that parallel the properties of the completion form used in many other functions.

However, specifying a query and index should suffice as a bare minimum to yield results:

curl -X 'GET' \
  'http://localhost:8082/api/v1/ai/search?query=snow&indexName=blogIndex' \
  -H 'accept: application/json'

The returned object will have the same schema as that returned by a viewtool search function, farther up the page.

The POST version looks for the same data, albeit in payload form rather than query parameters.

/api/v1/ai/search/related

A GET call to ai/search/related, calls for a sequence of query parameters that point the search function toward the relevant content already in the system, which will be used to perform a related-content search. The five parameters are: language, identifier, inode, indexName, and fieldVar — the first three pointing to a piece of content, the fourth to an index, and the fifth to a specific field within the contentlet.

curl -X 'GET' \  'http://localhost:8082/api/v1/ai/search/related?language=0&identifier=3d6fa2a4-2b48-4421-a2f5-b6518a7c0830&inode=ae49b86b-6bd2-47d4-924e-c8e747d9bf0a&indexName=blogIndex&fieldVar=blogContent' \
  -H 'accept: application/json'

The POST call follows the same pattern, albeit submitting its data in payload form instead of query parameters.

curl -X 'POST' 'http://localhost:8082/api/v1/ai/search/related' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "language":0,
  "identifier":"3d6fa2a4-2b48-4421-a2f5-b6518a7c0830",
  "inode":"ae49b86b-6bd2-47d4-924e-c8e747d9bf0a",
  "indexName":"blogIndex",
  "fieldVar":"blogContent"
}'

Embeddings Endpoints

/api/v1/ai/embeddings

The two methods associated with this path — POST and DELETE — either add or remove embeddings from the chosen index, without otherwise rebuilding or destroying the index.

POST utilizes the embeddings form properties in its payload data.

curl -X 'POST' \
  'http://localhost:8082/api/v1/ai/embeddings' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "query": "+contentType:blogIndex +variant:default +live:true",
  "limit": 1000,
  "offset": 0,
  "indexName": "blogIndex",
  "model": "gpt-3.5-turbo-16k",
  "fields":"blogContent"
}'

The DELETE version differs slightly. It still uses a payload, but its data members more closely resemble a related search, as it looks for indexName, identifier, inode, language, contentType, site, and deleteQuery — once again a Lucene query. As such, embeddings can be deleted on the basis of any of those — from individual contentlets to whole Content Types, to arbitrary query results.

/api/v1/ai/embeddings/db

DELETE call to this path can delete entire AI embeddings index database; use with care.

curl -XDELETE -k http://localhost:8082/api/v1/ai/embeddings/db \
-H "Content-Type: application/json" 

/api/v1/ai/embeddings/indexCount

GET call lists all indexes.

curl -X 'GET' \
  'http://localhost:8082/api/v1/ai/embeddings/indexCount' \
  -H 'accept: application/json'

Result excerpt:

{
    "indexCount":{
        "blogIndex":{
            "contentTypes":"Activity,Blog,calendarEvent,Image,webPageContent",
            "contents":42,
            "fragments":91,
            "tokenTotal":26797,
            "tokensPerChunk":294
        },
        ...,
        "cache":{
            "contentTypes":"cache",
            "contents":1,
            "fragments":80,
            "tokenTotal":21579,
            "tokensPerChunk":269
        }
    }
}

Embeddings Form

The schema of the payload object required for most embeddings operations takes the following form:

{
  "query": "string",
  "limit": int,
  "indexName": "string",
  "velocityTemplate": "string",
  "offset": int,
  "model": "string",
  "fields": "comma,separated,list,of,strings",
  "userId": "string"
}

Completions Endpoints

/api/v1/ai/completions

Called with POST, this endpoint accepts a payload containing the properties of the completion form.

curl -XPOST -k http://localhost:8082/api/v1/ai/completions \
-H "Content-Type: application/json" \
-d '{
  "prompt": "where do i vacation?",
"threshold":".2",
"searchLimit":50,
"stream": false,
"indexName": "blogIndex"
}'

It returns data in the form of the following full, untruncated completion operation data, containing the detailed prompt response.

{
    "timeToEmbeddings":"860ms",
    "total":5,
    "query":"What's the best place to vacation?",
    "threshold":0.25,
    "dotCMSResults":[
        {
            <content object properties>,
            "matches":[
                {
                    "distance":0.21085739135742188,
                    "extractedText":"<text excerpt>..."
                },
                ...
            ]
        },
        ...
    ],
    "operator":"<=>",
    "offset":0,
    "limit":50,
    "count":3,
    "openAiResponse":{
        "id":"chatcmpl-8rUy56hdxj210olTgFbNfVpxKVQla",
        "object":"chat.completion",
        "created":1707761117,
        "model":"gpt-3.5-turbo-16k-0613",
        "choices":[
            {
                "index":0,
                "message": {
                    "role":"assistant",
                    "content":"Based on the information provided, Fiji would be an excellent destination for a vacation. Fiji is located in the heart of the South Pacific and is blessed with 333 tropical islands. The islands are known for their luxurious..."
                },
                "logprobs":null,
                "finish_reason":"stop"
            }
        ],
        "usage":{
            "prompt_tokens":325,
            "completion_tokens":139,
            "total_tokens":464
        },
        "system_fingerprint":null
    },
    "totalTime":"2998ms"
}

Note that if stream is set to true, the results will arrive continuously in a sequence of many smaller data properties, containing about a word of content at a time:

data: {
    "id":"chatcmpl-8rTfxumekRWYLmDdCZwnLxlNcWAEO",
    "object":"chat.completion.chunk",
    "created":1707756149,
    "model":"gpt-3.5-turbo-16k-0613",
    "system_fingerprint":null,
    "choices":[
        {
            "index":0,
            "delta":{
                "content":" a"
            },
            "logprobs":null,
            "finish_reason":null
        }
    ]
}

/api/v1/ai/completions/rawPrompt

In addition to the above, completions/rawPrompt is an option that yields a completion response in a lighter-weight fashion, omitting the bulky dotCMSResults property and other context and metadata. The data members of the call are identical.

curl -X 'POST' 'http://localhost:8082/api/v1/ai/completions/rawPrompt' \
  -H 'accept: application/octet-stream' \
  -H "Content-Type: application/json" \
  -d '{
  "prompt": "testing costa rica for water",
  "searchLimit": 1000,
  "searchOffset": 0,
  "responseLengthTokens": 128,
  "language": 0,
  "stream": false,
  "fieldVar": "blogContent",
  "indexName": "blogIndex",
  "threshold": 0,
  "temperature": 1,
  "model": "gpt-3.5-turbo-16k",
  "operator": "<=>",
  "site": "demo.dotcms.com"
}'

The significantly leaner reply proceeds directly to the contents of what had been the openAiResponse property, above.

{
    "id":"chatcmpl-8rVNcDXxU8Z1BHwjpJLEfPJorrMHU",
    "object":"chat.completion",
    "created":1707762700,
    "model":"gpt-3.5-turbo-16k-0613",
    "choices":[
        {
            "index":0,
            "message":{
                "role":"assistant",
                "content":"When it comes to testing water in Costa Rica, there are a few important factors to consider. Costa Rica generally has good water quality, but it's always a good idea to take precautions to ensure that the water you consume is safe. Here are some options for testing water in Costa Rica..."
            },
            "logprobs":null,
            "finish_reason":"length"
        }
    ],
    "usage":{
        "prompt_tokens":13,
        "completion_tokens":128,
        "total_tokens":141
    },
    "system_fingerprint":null,
    "totalTime":"2475ms"
}

/api/v1/ai/completions/config

Sending a GET call to this path yields a JSON object describing the configuration of fundamental prompts and settings contained in the App configuration, and visible in the Config Values tab in the dotAI Tool.

curl -X 'GET' \
  'http://localhost:8082/api/v1/ai/completions/config' \
  -H 'accept: application/json'

A typical returned object will resemble this:

{
    "apiImageUrl":"https://api.openai.com/v1/images/generations",
    "apiKey":"*****",
    "apiUrl":"https://api.openai.com/v1/chat/completions",
    "availableModels":[
        "gpt-3.5-turbo",
        "gpt-3.5-turbo-16k",
        "gpt-4","gpt-4-1106-preview"
    ],
    "com.dotcms.ai.completion.default.temperature":"1",
    "com.dotcms.ai.completion.model":"gpt-3.5-turbo-16k",
    "com.dotcms.ai.completion.role.prompt":"You are a helpful assistant with a descriptive writing style.",
    "com.dotcms.ai.completion.text.prompt":"Answer this question\\n\\\"$!{prompt}?\\\"\\n\\nby using only the information in the following text:\\n\"\"\"\\n$!{supportingContent} \\n\"\"\"\\n",
    "com.dotcms.ai.debug.logging":"false",
    "com.dotcms.ai.embeddings.build.for.file.extensions":"pdf,doc,docx,txt,html",
    "com.dotcms.ai.embeddings.cache.size":"1000",
    "com.dotcms.ai.embeddings.cache.ttl.seconds":"600",
    "com.dotcms.ai.embeddings.delete.old.on.update":"true",
    "com.dotcms.ai.embeddings.minimum.file.size":"1024",
    "com.dotcms.ai.embeddings.minimum.text.length":"64",
    "com.dotcms.ai.embeddings.model":"text-embedding-ada-002",
    "com.dotcms.ai.embeddings.search.default.threshold":".25",
    "com.dotcms.ai.embeddings.split.at.tokens":"512",
    "com.dotcms.ai.embeddings.threads":"3",
    "com.dotcms.ai.embeddings.threads.max":"6",
    "com.dotcms.ai.embeddings.threads.queue":"10000",
    "configHost":"demo.dotcms.com (falls back to system host)",
    "imageModel":"dall-e-3",
    "imagePrompt":"Use 3:4 aspect ratio.",
    "imageSize":"1792x1024",
    "listenerIndexer":"{\n    \"blogIndex\":\"Blog.blogContent\"\n}",
    "model":"gpt-3.5-turbo-16k",
    "rolePrompt":"You are dotCMSbot, and AI assistant to help content creators generate and rewrite content in their content management system.",
    "textPrompt":"Use Descriptive writing style."
}

Completion Form

The schema of the payload object required for most completions operations or text generation takes the following form:

{
  "prompt": "string",
  "searchLimit": int,
  "searchOffset": int,
  "responseLengthTokens": int,
  "language": int,
  "stream": bool,
  "fieldVar": "string",
  "indexName": "string",
  "contentType": "comma,separated,list,of,strings",
  "threshold": float,
  "temperature": float,
  "model": "string",
  "operator": "string",
  "site": "string"
}