--- title: "Talking to other systems" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Talking to other systems} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r, include = FALSE} knitr::opts_chunk$set(collapse = TRUE, comment = "#>") library(blockr.process) ``` Half of what a real process waits for happens somewhere else: a file lands on an upload platform, a nightly load finishes, a ticket closes, a lab returns a result. None of that is a click in your app. The event log **is** the API. Current state is a fold over an append-only list of events, so causing an event is the only thing an outside system ever needs to do. ## The inbox One message per file, dropped into `/inbox/`: ```json {"instance": "2026Q1", "task": "delivery", "element": "Northgate", "field": "status", "value": "done", "actor": "delivery-platform"} ``` Only `task` is required. `instance` defaults to `"latest"`, `field` to `"status"`, `value` to `"done"`, `actor` to `"external"`, so the smallest useful message is `{"task": "delivery"}`. Nothing but a directory is involved: no server, no port, no token store, no dependency. Every language can write a file, and it works over any shared or synced mount. What it costs is that **the filesystem is the authentication**: whoever may write into the inbox may move a process forward. That is the same trust as "whoever may INSERT into the events table"; enforce it with directory permissions. ## End to end ```{r} store <- tempfile() process <- data.frame( task = c("each_unit", "delivery", "review", "approve"), name = c("for each unit", "Data delivery", "Review", "Approve"), role = c("", "system", "analyst", "management"), depends_on = c("", "", "delivery", "each_unit"), collection = c("unit", "", "", ""), parent = c("", "each_unit", "each_unit", "") ) start_instance(process, c("north", "south"), "2026Q1", store) ``` `delivery` is `system` **with no script**: the worker will not run it (no script) and the task list will not let anyone tick it (role `system`), so the only thing that can complete it is a message from outside. That is the convention for a task that waits on the outside world: BPMN's receive task. The platform delivers `north`: ```{r} write_inbox_message( store, task = "delivery", element = "north", instance = "2026Q1", actor = "delivery-platform", id = "delivery-2026Q1-north" ) ingest_inbox(store) ``` ```{r} tab <- instance_view(store, "2026Q1") tab[, c("task", "element", "status")] ``` `north` may be reviewed; `south` is still blocked, waiting for its own delivery. The event says who caused it and which message it was: ```{r} instance_log(store, "2026Q1")[1, c("task", "element", "value", "actor", "ref")] ``` `ref` is what makes an event traceable back **out** of the process: `actor` says which system spoke, `ref` says which of its messages this was. In production nobody calls `ingest_inbox()` by hand; `run_worker()` does it on every tick. A process with no scripts at all can run `ingest_inbox()` from cron instead; it is an ordinary function. ## Resending is free Networks time out and senders retry. A delivery counted twice is a wrong process, so **the file name is the idempotency key**: ```{r} write_inbox_message( store, task = "delivery", element = "north", instance = "2026Q1", actor = "delivery-platform", id = "delivery-2026Q1-north" ) ingest_inbox(store) sum(instance_events(store, "2026Q1")$task == "delivery") ``` One event, not two. Give a message a stable id that says what it is about (`delivery-2026Q1-north`), and a resend costs nothing. The duplicate is kept in `processed/` rather than deleted, because a file someone sent is evidence. ## Three outcomes | outcome | what happens | where the file goes | |---|---|---| | applied | one event is appended | `inbox/processed/` | | impossible | nothing is written | `inbox/failed/`, with a `.error` beside it | | early | nothing happens yet | stays in `inbox/` | "Impossible" means it can never apply, however often it is retried: no such task in this instance, an element that is not in the collection, an element on a task that does not repeat, unreadable JSON. The reason is written next to the file: ```{r} write_inbox_message(store, "delivery", element = "westside", instance = "2026Q1", id = "wrong") ingest_inbox(store, quiet = TRUE) readLines(file.path(inbox_dir(store), "failed", "wrong.json.error")) ``` "Early" means the message is right but the instance has not been started yet, a race rather than a mistake. The file stays where it is and the next tick picks it up. Validation is against **the instance**, not a schema: the definition an instance was started with is stamped into its log, so the check knows which tasks exist and which elements are real. ## Sending from somewhere else Anything that can write a file can send a message. Write to a temporary name in the same directory and rename, so a half-written file is never read: ```sh # shell cat > /srv/store/inbox/.tmp <