Using Structured output

What is Structured output?

Asking question on web based client works really well for human interaction. However when working programmatically you may want the response to be in a specific format, one that can we easily mapped to a class or struct, allowing you to use the stand language patterns of deserializing json into an object/class.

Structured output allows you to define a json schema that the output will adhere too.

Tutorial

For ease and to provide clarity we are going to just use 'Direct Calling', rather than function calling or RAG for example. However structured output can be added to any of the other patterns.

So the question we are going to ask is

Query

Tell me about the game Pang in under 100 words

running this as is gives the following response, in the standard output format:

{
    "answer": "Pang, also known as \"Buster Bros\", is a classic arcade game where you use a cannon to shoot at bouncing balls. Each ball is a \"pang\", and you must break it into two smaller pongs by hitting it at the right spot.  The pongs become progressively smaller and faster until they disappear.  As you progress, the difficulty increases with more balls, new types of balls, and even balls that ricochet.  The goal is to clear the screen of all pongs before they reach the bottom. \n",
    "query": "Tell me about the game Pang in under 100 words",
    "spanId": "OtukXDONg",
    "messages": null
}

To use structured output (when using REST) you need to create a jsonschema of the expected output. For more information on the Json Schema standard see here.

JellyFaaS (for ease of use base64 encodes the request and response) rather than embedding json into json.

For the the above we require the following format:

{
  "type": "object",
  "properties": {
    "question": {
      "type": "string",
      "description": "The prompt asked, this is required"
    },
    "output": {
      "type": "string",
      "description": "The response to the question"
    },
    "wordCount": {
      "type": "string",
      "description": "The count of the number of words, this is required"
    }
  },
  "required": [
    "question",
    "output",
    "wordCount"
  ]
}docs 

`JsonSchema` is quite easy to understand, basically in this simple example we are describing the three fields we need back:

* The Question
* The Response to the question
* The work count.

We are then specified that these are all required.

Once we base64 encoded:

```shell title="base64 encoded jsonschema"
eyJ0eXBlIjoib2JqZWN0IiwicHJvcGVydGllcyI6eyJxdWVzdGlvbiI6eyJ0eXBlIjoic3RyaW5nIiwiZGVzY3JpcHRpb24iOiJUaGUgcHJvbXB0IGFza2VkLCB0aGlzIGlzIHJlcXVpcmVkIn0sIm91dHB1dCI6eyJ0eXBlIjoic3RyaW5nIiwiZGVzY3JpcHRpb24iOiJUaGUgcmVzcG9uc2UgdG8gdGhlIHF1ZXN0aW9uIn0sIndvcmRDb3VudCI6eyJ0eXBlIjoic3RyaW5nIiwiZGVzY3JpcHRpb24iOiJUaGUgY291bnQgb2YgdGhlIG51bWJlciBvZiB3b3JkcywgdGhpcyBpcyByZXF1aXJlZCJ9fSwicmVxdWlyZWQiOlsicXVlc3Rpb24iLCJvdXRwdXQiLCJ3b3JkQ291bnQiXX0=

Information

Websites such as https://www.base64decode.org/ exist to help create base64 string, or you can use the JellyFaaS CLI or any library included in your language of choice.

Now we add this to the request:

Request with structured output

{
    "query": "Tell me about the game Pang in under 100 words",
    "structured_output":"eyJ0eXBlIjoib2JqZWN0IiwicHJvcGVydGllcyI6eyJxdWVzdGlvbiI6eyJ0eXBlIjoic3RyaW5nIiwiZGVzY3JpcHRpb24iOiJUaGUgcHJvbXB0IGFza2VkLCB0aGlzIGlzIHJlcXVpcmVkIn0sIm91dHB1dCI6eyJ0eXBlIjoic3RyaW5nIiwiZGVzY3JpcHRpb24iOiJUaGUgcmVzcG9uc2UgdG8gdGhlIHF1ZXN0aW9uIn0sIndvcmRDb3VudCI6eyJ0eXBlIjoic3RyaW5nIiwiZGVzY3JpcHRpb24iOiJUaGUgY291bnQgb2YgdGhlIG51bWJlciBvZiB3b3JkcywgdGhpcyBpcyByZXF1aXJlZCJ9fSwicmVxdWlyZWQiOlsicXVlc3Rpb24iLCJvdXRwdXQiLCJ3b3JkQ291bnQiXX0="
}

This returns:

Response

{
    "answer": "eyJvdXRwdXQiOiAiUGFuZyBpcyBhbiBhcmNhZGUgZ2FtZSB3aGVyZSB5b3UgdXNlIGEgcGFkZGxlIHRvIGJvdW5jZSBiYWxscyBhdCBncm91cHMgb2YgY29sb3JmdWwgYnViYmxlcy4gSGl0dGluZyBhIGJ1YmJsZSBzcGxpdHMgaXQgaW50byBzbWFsbGVyIGJ1YmJsZXMsIGFuZCB0aGUgZ29hbCBpcyB0byBjbGVhciB0aGUgc2NyZWVuLiBUaGUgZ2FtZSBpcyBrbm93biBmb3IgaXRzIGFkZGljdGl2ZSBnYW1lcGxheSwgY2hhbGxlbmdpbmcgbGV2ZWxzLCBhbmQgZGlzdGluY3RpdmUgc291bmQgZWZmZWN0cy4iLCAicXVlc3Rpb24iOiAiVGVsbCBtZSBhYm91dCB0aGUgZ2FtZSBQYW5nIGluIHVuZGVyIDEwMCB3b3JkcyIsICJ3b3JkQ291bnQiOiAiNjAifQ==",
    "query": "Tell me about the game Pang in under 100 words",
    "spanId": "dG1a_DdHg",
    "messages": null
}

Decoding this answer gives us:

Structured output response

{
  "output": "Pang is an arcade game where you use a paddle to bounce balls at groups of colorful bubbles. Hitting a bubble splits it into smaller bubbles, and the goal is to clear the screen. The game is known for its addictive gameplay, challenging levels, and distinctive sound effects.",
  "question": "Tell me about the game Pang in under 100 words",
  "wordCount": "60"
}