--- title: "Get started with blockr.assistant" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Get started with blockr.assistant} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>", eval = FALSE ) ``` `blockr.assistant` mounts an LLM chat panel alongside a [`blockr.dock`](https://github.com/BristolMyersSquibb/blockr.dock) board. The chat is powered by [`ellmer`](https://ellmer.tidyverse.org/) and [`shinychat`](https://posit-dev.github.io/shinychat/); the assistant sees the board's current state through a refreshed-each-turn system prompt and can read or modify the board via a registered tool surface. ## Mount it on a board ```{r} library(blockr.core) library(blockr.dock) library(blockr.assistant) board <- new_dock_board( blocks = c( data = new_dataset_block("iris"), head = new_head_block(n = 10L), plot = new_scatter_block(x = "Sepal.Length", y = "Sepal.Width") ), links = c( new_link("data", "head", "data"), new_link("head", "plot", "data") ), extensions = list(assistant = new_assistant_extension()) ) serve(board) ``` That's the whole setup. The assistant takes its chat client from the board's `llm_model` option (resolved by `blockr.core`'s `default_chat` unless overridden), registers the read and mutation tools, and starts the chat module. ## Talk to it The system prompt the model receives on each new turn contains three sections: a fixed intro / conventions block, the auto-generated tool catalogue, and a compact board summary. Trivial questions land without a tool round-trip: > **You**: How many blocks are on the board? > > **Assistant**: There are three blocks: `data` (`dataset_block`), > `head` (`head_block`), and `plot` (`scatter_block`). Richer questions about a specific block trigger one drill-down call: > **You**: What is `n_rows` on the head block? > > **Assistant**: \[calls `describe_block(id = "head")`\] `n_rows` is > currently 10. Questions about what a block holds go through the `inspect_results` tool, which evaluates R against an environment with every committed block result bound by id: > **You**: What are the unique species in the data block? > > **Assistant**: \[calls `inspect_results(code = "unique(data$Species)")`\] > Three: setosa, versicolor, virginica. Anything the code draws comes back as an image. The call runs with an off-screen graphics device open, so a `plot()` call, an auto-printed ggplot or lattice object, or an auto-printed plot recording all return the rendering alongside the text output -- one image per page drawn, sized by the `width` and `height` the model asks for. Only base R is attached, so the model prefixes calls from other packages (`graphics::hist()`, `ggplot2::ggplot()`) -- the same thing it would write inside a code block. A plot block's result is a list of recordings, drawn with `evaluate::replay()`, which is the call the board renders it with. Mutations stage across the turn and flush atomically when the assistant finishes: > **You**: Add a filter for `Sepal.Width > 3` between head and plot. > > **Assistant**: \[stages `add_block(...)`, `add_link(...)`, > `add_link(...)`, flushes once\] Added. ## Configure the model The chat backend comes from a global option: ```{r} library(ellmer) blockr.core::blockr_option( "chat_function", function(system_prompt = NULL, params = NULL) { chat_anthropic(model = "claude-sonnet-4-5", system_prompt = system_prompt) } ) ``` Any `ellmer::chat_*()` constructor that follows the standard `(system_prompt, params)` shape will do. See [ellmer's provider matrix](https://ellmer.tidyverse.org/) for options and credential handling. ## Customise the prompt `new_assistant_extension(system_prompt = ...)` accepts either: * a **string** -- used verbatim as a static prompt, no refresh, no auto-appended board context. You take full control: ```{r} new_assistant_extension( system_prompt = "You are a terse SQL coach. Refuse non-SQL." ) ``` * a **function** -- called each refresh with `(board, client, ...)`, returning the prompt string. The default `default_system_prompt` composes the four-section default. To extend it, wrap and call through. Custom functions should accept `...` for forward-compatibility with future phases adding inputs. ```{r} my_prompt <- function(board, client, ...) { paste( "ADDITIONAL RULE: prefer modify_block over remove + add", "when possible.", "", default_system_prompt(board, client, ...), sep = "\n" ) } new_assistant_extension(system_prompt = my_prompt) ``` You can call `default_system_prompt()` with no arguments at the REPL to inspect just the intro block. ## Teach it your conventions with skills A block type's `guidance` is the package author's view of how to configure it. A **skill** is the deployment's view -- that a particular flexible block is, here, preferably used a certain way. Skills are markdown files on disk, so a domain expert who does not write R can contribute one, and they cost nothing per turn until the model reads them. A skill is a directory whose name is the skill name; `SKILL.md` is the entry point and sibling files are optional bundled resources the model pulls only when it needs them. ``` skills/ adam-exposure/ SKILL.md column-conventions.md ``` `SKILL.md` is YAML frontmatter plus a markdown body: ```` --- name: adam-exposure description: >- How exposure tables are built here. Read before configuring a block over ADEX. blocks: - dplyr_filter_block requires: - blockr.dplyr (>= 0.2.0) --- Always filter to `SAFFL == "Y"` before summarising; see `column-conventions.md` for the column glossary. ```` Only `name` (which must match the directory and hold nothing but lowercase letters, digits and hyphens) and `description` are required. Everything else is optional: | Field | Behaviour | |---|---| | `requires` | R dependency specs; unmet means the skill does not exist for the session | | `blocks` | block type ids this skill is scoped to | | `extensions` | extension classes this skill is scoped to | | `user-invocable` | also registers a `/name` slash command | | `disable-model-invocation` | hide it from the model entirely: no catalogue entry anywhere, and `read_skill` refuses it | Unknown fields are ignored, so a skill authored for another agent drops in unchanged. `allowed-tools` is the exception: it is warned about and not honoured, because every mutation here already funnels through staging and an explicit commit. `description` is the only always-on model-facing text, so keep it to what the model needs in order to decide whether to read the skill. One description is truncated at `blockr.assistant_skill_description_max_chars` (default 1024); the always-on cost is all of them together, so the assistant also logs a warning once the global catalogue passes `blockr.assistant_skill_catalogue_max_chars` (default 4000). Scoping a skill to the blocks or extensions it bears on takes it off that budget entirely. **Where they surface.** A skill scoped to neither blocks nor extensions is global: it appears in the `## Skills` section of the default system prompt. A block-scoped skill appears in `describe_block_type()` and `describe_block()`, next to the block's `guidance` and taking precedence over it; an extension-scoped one appears in `describe_extension()`. Either way the model loads the body with the `read_skill` tool once it decides the skill is relevant, and a `user-invocable` skill can also be run by name from the chat input -- `/adam-exposure summarise by treatment arm`. **Where they come from.** The package ships one skill (`layout`, the view and panel grammar). Point the assistant at your own directory with an option or the matching environment variable: ```{r} blockr.core::blockr_option("assistant_skills", "/srv/blockr/skills") ``` A deployment skill shadows a built-in of the same name. A directory that does not exist is an error at mount rather than a silently inert configuration. Edits are picked up without restarting the app. **Skills are instructions, not capabilities.** A skill carries system-prompt weight: writing one is editing the assistant's behaviour, not adding documentation. It cannot widen the tool surface, and the invariants that matter are enforced in code -- nothing applies to the board without a commit, whatever any skill says. Authorship is operator-only by construction: the directory comes from a server-side option, never from board state, and no tool writes a skill. ## Give it your own tools A skill shapes how the assistant uses the tools it has, but it cannot add one. To let the model reach something the board does not hold, such as a study catalogue, write the tool with `ellmer::tool()` and list it in the `blockr.assistant_tools` option, in the script that starts the app: ```{r} studies <- data.frame( id = c("ABC-101", "ABC-102"), title = c("Phase II dose finding", "Phase III efficacy") ) find_studies <- ellmer::tool( function(query) { hits <- studies[grepl(query, studies$title, ignore.case = TRUE), ] paste(hits$id, hits$title, sep = ": ", collapse = "\n") }, name = "find_studies", description = "Find the studies whose title matches a pattern.", arguments = list( query = ellmer::type_string("Regular expression to match titles.") ) ) options(blockr.assistant_tools = list(find_studies)) ``` Every chat client the assistant builds gets these tools next to its own, including the one built when the user switches models. The chat titles them as it titles its own, so `find_studies` shows as "Find studies" unless the tool sets a `title` through `ellmer::tool_annotations()`. A tool may not take a name the assistant already uses, and two tools in the list may not share one: whichever is registered second would replace the first, leaving one of the assistant's tools or one of yours unreachable. Either mistake stops the chat from being built, with an error in the app naming the tool. The assistant's names include the `add_` and `modify_` it registers for each block type as the model asks for them. **Tools are capabilities.** A tool is R code running in the app's process, with whatever access that process has. Like a skill, it can come only from a server-side option: a board file cannot carry one, and a saved board does not take yours along. The option has no environment-variable form, as a tool is an R object rather than a string. A tool is handed the arguments the model passes and not the board, so the board still changes only through a commit. The same tool objects serve every session of the app, so a tool that keeps state between calls shares it between users. ## Extend per block class The package exports two S3 generics that let block / stack authors customise the detailed descriptions the model can request on demand: | Generic | Used by | When you'd override | |---|---|---| | `describe_block(x, board, id)` | `describe_block` tool | full description on user demand | | `describe_stack(x)` | `describe_stack` tool | ditto for stacks | The default methods report the class, name, arguments, external control and incoming links; override them when your class carries extra state worth surfacing. The compact one-line-per-entity board summary in the dynamic system prompt is produced internally, off blockr.core's `str_value()` generic -- the value-returning compact counterpart to `str()`, owned below this package -- not an assistant-specific generic. To influence how your block, stack or layout renders there, define a `str_value()` method in its home package: the correct extension point and dependency direction (the assistant sits above core / dock / dag, so it cannot own a generic those packages would implement). A `dock_stack`'s colour reaches the summary exactly that way. A worked example overriding the detailed description for a bespoke block class: ```{r} describe_block.fancy_block <- function(x, board, id, ...) { c( NextMethod(), "", sprintf("Rule semantics: %s", attr(x, "rule_doc")) ) } ``` ## Where to go next * [Roadmap](design/0-roadmap.html) for the design pillars of the initial release. * The per-phase design notes under [Design notes](https://bristolmyerssquibb.github.io/blockr.assistant/) for contributors and for understanding the architectural decisions. * [Issue tracker](https://github.com/BristolMyersSquibb/blockr.assistant/issues) for feature requests and bug reports.