AI

This feature provides an AI agent for Grid. It provides a chat panel that allows the user to send messages to the agent and see the responses. Which allows the user to use natural language to interact with the Grid in different ways. For example, the user can ask the agent to filter the data, sort it, or select specific records. The agent can also perform data manipulation operations such as adding, updating, deleting records.

To use this feature, you need to provide a promptUrl to which the agent will send the prompts. On the other end of this URL, there should be a service that adds credentials to the request and forwards the prompt to the LLM API of your choice.

You also need to configure a apiPlugin which ensures the prompt format matches the API you are using.

new Grid({
  features : {
     ai : {
        promptUrl : '/ai/prompt',
        apiPlugin : OpenAIPlugin,
     }
  }
})

This is the minimum configuration required to get started, but you may also want to configure the following:

  • model - Adds a model property to the prompt sent to the LLM API which is often required
  • chatButton - Change the appearance of the chat button and the chat panel
  • tools - Add custom tools to the agent's tool list, or remove existing unwanted tools
  • userSettings - Provide the current user's settings
  • userSettingsUpdate - To listen for changes to the user's settings and save them

And for voice input/output, check out:

But these are not all configurations available. Check out the demo linked in the right panel for a full configuration example. And make sure to read this documentation to learn more about all available configurations.

This feature is disabled by default

Configs

38

Common

disabledInstancePlugin
listenersEvents

Other

AI plugin to use. Provide the class here. Available plugins are:

  • OpenAIPlugin - use OpenAI API
  • AnthropicPlugin - use Anthropic API
  • GooglePlugin - use Google API

You can also provide a custom plugin by extending the AbstractApiPlugin class.

Shows an AI model selector in the SettingsPanel where the user can select between the available API:s

Set this to true or a config object to add a ChatButton

Set to true to automatically console.log things that happen internally. AI responses and audio processing for example.

An array of strings which will be circulated as waiting status texts when awaiting long AI responses Defaults to ['Waiting', 'Waiting ...', 'Still waiting', 'Still waiting ...']

The URL to send feedback (thumbs up/thumbs down) to

Additional options to pass to fetch requests

By default, data item which are targeted in an AI action will be highlighted in the UI. Set this to false to prevent this behavior.

Set to false to not manage chat history on the client

Max tokens setting (if applicable)

model: StringAIBase

AI model name (if applicable)

Use this to provide more details of data models currently available for the AI agent to work with. Provide an object with model class names as keys and configuration objects as values.

A setup in a Bryntum Grid application which uses the class Project for its rows and also have a related class Member could look like this:

{
    Member  : { store : memberStore },
    Project : {
        relations : {
            members : {
                relatedModelName : 'Member'
            },
            owner : {
                relatedModelName : 'Member'
            }
        }
    }
}

The URL to send text prompts to

redo: Boolean= trueAIBase

Enables support for redoing data modifying actions performed by the AI. Shows a redo icon under the last redoable message in the ChatPanel. Requires undo to be enabled.

Set to true to always prompt the user for confirmation before allowing the AI to add records

Set to true to always prompt the user for confirmation before allowing the AI to remove records

Set to true to always prompt the user for confirmation before allowing the AI to update records

Set to true to upon creation, send a test prompt to verify the connection to the AI backend. If the test fails, a warning icon will be shown on the chat panel.

Temperature setting (if applicable). Overrides the default temperature set by the apiPlugin.

The URL to send text for speech synthesizing

timeout: Number= 60000AIBase

A number of milliseconds a prompt is allowed to take before being aborted. Defaults to 60000 (1 minute).

tools: Object<String, AIToolConfig>

Use this to add custom tools to the AI model's tool list or to remove existing tools. Provide an object with the tool names as keys and configuration objects as values. Please note that modifying existing tools is not supported, only removing them.

Specific tools for Grid AI are:

  • getRecords - Used by agent to retrieve records from the store
  • updateRecords - Updates records in the store
  • deleteRecords - Deletes records from the store
  • addRecord - Add a new record in the store
  • filterRecords - Filters the Grid's store
  • unfilterRecords - Removes all filters from the Grid's store
  • sortRecords - Sorts the Grid's store
  • selectRecords - Selects records in the Grid
  • lockRows - Locks rows to the top (LockRows feature required)
  • reloadGrid - Reloads data. Only available if the Grid can be reloaded

Also, the tools from tools are available

Tool names will change according to your grid row model $name property. For example, if your row model is named Person, the tool names will be getPersons, updatePersons, etc.

Use this to provide example prompt scenarios to "teach" the AI of more complex tasks. Must be configured as an array containing arrays which contains the AI messages for the complete conversation.

Please be advised that the content here will be provided in each request sent to the AI API and will therefore consume tokens and take up space in the context window.

Adding an example to the "training data" will often solve one particular case, but may "confuse" the AI in ways not expected.

The URL to send audio for transcribing

undo: Boolean= trueAIBase

Enables support for undoing data modifying actions performed by the AI. Shows an undo icon under the last undoable message in the ChatPanel.

A configuration object containing the user's preferred AI feature settings. If there is a matching AI feature class config, the user setting will take precedence JSON.

Experimental configuration of the AI verbosity. Valid values are "1", "2" and "3", where "1" is lowest.

Configuration settings for voice activation

voiceOnly: Boolean= falseAIBase

Forces voice only mode, which hides the chat panel and only shows a record chat button.

Misc

clientInstancePlugin
localeClassLocalizable
localizableLocalizable

Properties

17

Common

disabledInstancePlugin

Class hierarchy

isAI: Boolean= truereadonly
Identifies an object as an instance of AI class, or subclass thereof.
isAI: Boolean= truereadonlystatic
Identifies an object as an instance of AI class, or subclass thereof.
isAIBase: Boolean= truereadonlyAIBase
Identifies an object as an instance of AIBase class, or subclass thereof.
isAIBase: Boolean= truereadonlystaticAIBase
Identifies an object as an instance of AIBase class, or subclass thereof.
isEventsEvents
isInstancePluginInstancePlugin
isLocalizableLocalizable

Lifecycle

configBase

Misc

clientInstancePlugin
localeHelperLocalizable
localeManagerLocalizable

Other

Functions

28

Configuration

applyDefaultsstaticBase

Events

Lifecycle

destroystaticBase

Misc

doDisableInstancePlugin
initClassstaticBase
isOfTypeNamestaticBase
mixinstaticBase
optionalLstaticLocalizable

Other

LstaticLocalizable
onEvents
relayAllEvents
triggerEvents
unEvents

Events

6

Fired when the user changes it's settings via the settings panel

// Adding a listener using the "on" method
aI.on('userSettingsUpdate', ({ userSettings, changes }) => {

});
ParameterTypeDescription
userSettingsAISettingsModel

The user settings record

changesObject

The changed properties and the new value

catchAllEvents
destroyEvents
disableInstancePlugin
enableInstancePlugin

Event handlers

6

Called when the user changes it's settings via the settings panel

new AI({
    onUserSettingsUpdate({ userSettings, changes }) {

    }
});
ParameterTypeDescription
userSettingsAISettingsModel

The user settings record

changesObject

The changed properties and the new value

onDestroyEvents
onDisableInstancePlugin
onEnableInstancePlugin

Typedefs

7

Configuration object used in the AI feature models config

ParameterTypeDescription
storeStore

The store instance for this model

relationsObject<String, AIRelatedModelConfiguration>

Relations configuration. Provide an object with the relation properties as keys and a configuration object as value

Configuration object for the AI feature's availableApis config. All properties are required.

ParameterTypeDescription
idString

The AI model code, will be sent to the promptUrl

nameString

The AI model display name, will be used in AI model selector

apiPluginClass

The apiPlugin which is associated with this model

Configuration object used in the AIModelConfiguration

ParameterTypeDescription
relatedModelNameString

The related model class name

descriptionString

A description which will be included in the model details provided to the AI agent

Tool configuration object

ParameterTypeDescription
argumentsObject
arguments.argsObject

The arguments provided to the tool invocation

ParameterTypeDescription
descriptionString

A quite short description of the tool's purpose for the AI agent

availableBoolean | function

Whether or not the tool is available at the time of the request. If a function is provided, it must return a Boolean.

fnfunction

The function to call when the tool is invoked

parametersObject<string, AIToolProperty>

The parameters schema of the tool. The keys are the parameter names and the values are objects describing that parameter

A tool parameters schema object

ParameterTypeDescription
typeString

The type of the property: object, string, number, boolean, array

descriptionString

A short description of the property's purpose for the AI agent

propertiesObject<String, AIToolProperty>

If the property is an object, this describes the properties of the object.

requiredString[]

If the property is an object, this describes the required properties of the object

itemsAIToolProperty

If the property is an array, this describes the items of the array

Configuration object used in the AI feature's voiceActivation config

ParameterTypeDescription
conversationTimeoutNumber

The time in milliseconds a voice conversation is allowed to be active. Defaults to 10 minutes.

replyOnActivationBoolean

Set to true to have the AI reply immediately when the phrase is recognized, without waiting for further user input.

languageString

The language code for the native speech recognition

autoStartDialogConfigObject

Configuration object for the dialog asking the user to enable voice activation on page load

phraseString

The phrase to listen for to activate voice control

autoStartBoolean

Set to true to start voice activation on page load

useNativeSpeechRecognitionBoolean

Set to true to use the browser's native speech recognition (if available). At this time, this is required to be set to true to get voice activation to work.