Build a Reliable AI Agent in R with Tool Calling and Verified Data

Nov 02, 2026 342 views
[This article first appeared on the [Jumping Rivers Blog](https://www.jumpingrivers.com/blog/agents-in-production-first-agent/) and was republished by [R-bloggers](https://www.r-bloggers.com/). You can [report a content issue](https://www.r-bloggers.com/contact-us/). To contribute to R-bloggers, [submit your blog](https://www.r-bloggers.com/add-your-blog/) or use [R Posts](http://r-posts.com/) if you don’t have one. [![Article image](https://www.jumpingrivers.com/blog/agents-in-production-first-agent/)](https://www.jumpingrivers.com/blog/agents-in-production-first-agent/) This is the first of five posts about putting AI agents into production with R. The series builds one clinical trial safety assistant from the ground up: first moving from chatbot to agent, then adding a Shiny app, checking reliability, covering audit trails, costs and models, and finally sharing tools through MCP. The problem starts with a deceptively simple question: how many people taking the high dose of a drug developed itching in a clinical trial? Asked twice, Gemini returned 21 and 25. Both responses sounded confident and cited trial data files the model had never been given. Those answers came from its training memory, so there was no way to verify what it had counted. That gap between fluency and evidence is the point. In this series, the model chooses an R function and R performs the count. The model is told not to do arithmetic itself; a later installment tests whether it follows that instruction. Even then, the model can ask the wrong question, so each result reports the filters and population behind the number. You don’t need a pharmaceutical background to follow the example. The same design applies whenever a team asks questions about data. Later posts add the Shiny interface, tests, controls over access to patient records, an audit trail, and a way for other AI agents to use the tools.

What makes an AI agent different?

A chatbot generates answers from patterns learned during training. An agent can also request actions from your software through tool calling, sometimes called function calling. You give the model tools with names, descriptions, and argument definitions. If a question calls for one, the model sends a request such as `count_subjects(arm = "Placebo")`. Your R code runs the function, returns its result, and lets the model decide whether it needs another tool call or can answer. That repeated exchange is the agent loop. The boundary matters: **the model never executes your code.** It can ask for a function; your R session controls what runs and what data comes back.
The agent loop: the model requests tools, R runs them, and the model uses the returned results to answer.
Diagram of the agent loop. Your question goes to the model. The model sends a tool request, such as count_subjects with arm Placebo and term PRURITUS, to your R session. R runs the function on the data and sends back a tool result, such as 8 subjects with the event. This repeats until the model can answer, and it is told to answer from the tool results only.
The R package {ellmer} handles that exchange. You write regular R functions, describe them, then register them with a chat. If you’re new to {ellmer}, the [LLMs in R & Python webinar](https://www.jumpingrivers.com/blog/programming-llms-r-python/) is a useful starting point. The [{ellmer} tool-calling guide](https://ellmer.tidyverse.org/articles/tool-calling.html) also recommends keeping exact computations inside the tools.

Setup and access

Download or clone [the part one folder in the blog repository](https://github.com/jumpingrivers/blog/tree/main/blogs/agents-in-production/part-1), then open it as a project in R 4.5. The project uses {renv} to lock package versions:
install.packages("renv")
renv::restore()
You can install the packages directly instead, though those versions may be newer than the ones used for the article:
install.packages(c("ellmer", "pharmaverseadam", "dplyr", "jsonlite"))
You’ll need a Gemini API key. The [alt-text post](https://www.jumpingrivers.com/blog/ellmer-dynamic-alt-text/) explains how to obtain one. To save it for the project, run `file.edit(".Renviron")`, add `GEMINI_API_KEY=your-key`, save, and restart R. The project README includes additional setup guidance, particularly for Windows. The free API tier works for this public trial data. It isn’t suitable for patient or confidential data. Google’s terms also require a paid service for apps offered to users in the UK, EEA, or Switzerland; the series returns to that issue in part four. The code was tested in September 2026 with R 4.5.1, {ellmer} 0.5.0, and {pharmaverseadam} 1.3.0. {ellmer} changes quickly, so version differences may explain different behavior.

The trial data and the choices behind each count

The example uses CDISC pilot study 01, a sample study published by CDISC, the organization that sets clinical trial data standards. It tested a xanomeline skin patch in people with mild to moderate Alzheimer’s disease. Participants, called *subjects* in the datasets, received placebo, a low dose, or a high dose over 26 weeks. The {pharmaverseadam} package reconstructs the study’s analysis data in CDISC ADaM format. The example needs two datasets: `adsl`, with one row per subject, and `adae`, with one row per adverse event. An adverse event is an unwanted medical occurrence during the trial, whether or not the treatment caused it. Here’s where a count can change before the model ever sees the data. The `adsl` dataset has 306 people, but 52 are screen failures. The safety population is therefore 254, selected with `SAFFL == "Y"`. Treatment assignment also differs from treatment received. Each subject has a planned arm (`ARM`) and a received treatment (`ACTARM`). Twelve people assigned to high dose left within two weeks, before moving up to 81 mg, so {pharmaverseadam} records them in the low-dose group.
Planned arms versus treatment received: 86, 84, and 84 subjects become 86, 96, and 72.
Diagram. Everyone in adsl, 306 people, minus 52 screen failures, leaves a safety population of 254 subjects. Split by planned arm, that is 86 placebo, 84 low dose and 84 high dose. Split by treatment received, as {pharmaverseadam} records it and our tools count it, it is 86 placebo, 96 low dose and 72 high dose. An arrow shows 12 subjects moving from the planned high dose to the low dose, because they stopped within two weeks, before their dose went up to 81 mg. A note says this is how {pharmaverseadam} records them, and that the pilot's own analysis kept them on the high dose.
The pilot’s original analysis kept those 12 people in the high-dose group, while this example follows the treatment received as recorded by {pharmaverseadam}. If you work with trial data, the [note for pharma readers](https://www.jumpingrivers.com/blog/agents-in-production-first-agent/#a-note-for-pharma-readers) discusses choices a formal analysis plan might handle differently. The tools also restrict events to treatment-emergent ones, using `TRTEMFL == "Y"`. That leaves 1,122 of 1,191 event rows. One function in `R/data.R` applies these rules:
ARMS <- c("Placebo", "Xanomeline Low Dose", "Xanomeline High Dose")
load_trial_data <- function() {
subjects <- pharmaverseadam::adsl |>
dplyr::filter(SAFFL == "Y") |>
dplyr::select(USUBJID, ACTARM, AGE, SEX, RACE)
events <- pharmaverseadam::adae |>
dplyr::filter(TRTEMFL %in% "Y", USUBJID %in% subjects$USUBJID) |>
dplyr::select(USUBJID, ACTARM, AEDECOD, AETERM, AEBODSYS, AESER, AESEV, ASTDT)
list(
subjects = subjects,
events = events,
# Every term and body system recorded in the trial, including the few
# never treatment-emergent, so asking about them gives a real zero.
terms = sort(unique(pharmaverseadam::adae$AEDECOD)),
body_systems = sort(unique(pharmaverseadam::adae$AEBODSYS))
)
}

What happens when you ask without tools

The authors asked `gemini-3.7-flash` the same question twice, without giving it any tools:
# Without tools, the model can only guess.
bare <- ellmer::chat_google_gemini()
bare$chat("In CDISC pilot study 01, how many subjects on high-dose xanomeline had pruritus?", echo = "output")
Pruritus means itching. The answers below are reproduced as returned, including their Markdown, with headings bolded.
Pedro
In CDISC pilot study 01, how many subjects on high-dose xanomeline had pruritus?
Gemini, first try

In the CDISC Pilot Study 01 (Safety Pilot), 21 subjects (or 25.0%) in the Xanomeline High-Dose group had the adverse event pruritus.

Summary Breakdown for Pruritus (Treatment-Emergent AEs):

  • Placebo ($N = 86$): 2 subjects (2.3%)
  • Xanomeline Low Dose (54 mg) ($N = 84$): 11 subjects (13.1%)
  • Xanomeline High Dose (81 mg) ($N = 84$): 21 subjects (25.0%)

(Source: CDISC Pilot 01 Table 14-4.01 / ADAE dataset, Preferred Term: Pruritus under the System Organ Class “Skin and subcutaneous tissue disorders”)

Gemini, second try

In the CDISC Pilot Study 01 (evaluating Xanomeline vs. Placebo in Alzheimer’s Disease), 25 subjects in the Xanomeline High Dose treatment group experienced Pruritus (MedDRA Preferred Term: PRURITUS).

Context & Breakdown across Treatment Arms:

  • Placebo ($N = 86$): 5 subjects (5.8%)
  • Xanomeline Low Dose ($N = 84$): 18 subjects (21.4%)
  • Xanomeline High Dose ($N = 84$): 25 subjects (29.8%)

(Source: CDISC SDTM/ADaM Pilot Project 01 Adverse Events Analysis Dataset, ADAE)

Neither response matches the count produced from the data, though the high-dose figures each resemble a real result under a different interpretation: | Result | High-dose pruritus | Explanation | |---|---:|---| | First answer | 21 of 84 (25.0%) | The low-dose count divided by the planned high-dose group size | | Second answer | 25 of 84 (29.8%) | The right count divided by the planned high-dose group size | | Data, by treatment received | 25 of 72 (34.7%) | Treatment-emergent events among the received high-dose group | For placebo, the data gives 8 of 86 (9.3%); the other figures in Gemini’s two breakdowns match nothing. The second response is the one most likely to pass a casual review: its numerator is right, and only the denominator is wrong. Neither response explains its population, treatment assignment, or event filters. That’s a more serious weakness than a wrong number that looks obviously invented.

Building tools the model can use

The model sees each tool’s description, not its R code. That description helps it decide when to call the function. Most of the counting logic sits in `count_subjects()` in `R/tools.R`:
count_subjects <- function(data, arm = NULL, serious_only = FALSE, term = NULL, body_system = NULL) {
arm <- check_arm(arm)
serious_only <- check_flag(serious_only, "serious_only")
term <- check_known(
check_string(term, "term"), data$terms, "preferred term",
"Call ae_summary to see the terms used in this trial."
)
body_system <- check_known(
check_string(body_system, "body_system"), data$body_systems, "body system",
"Call ae_summary with by = 'body_system' to see them."
)
data <- filter_arm(data, arm)
events <- data$events
if (serious_only) events <- dplyr::filter(events, AESER == "Y")
if (!is.null(term)) events <- dplyr::filter(events, AEDECOD == term)
if (!is.null(body_system)) events <- dplyr::filter(events, AEBODSYS == body_system)
n <- dplyr::n_distinct(events$USUBJID)
total <- nrow(data$subjects)
jsonlite::toJSON(list(
arm = arm %||% "All arms",
# Say what was counted, so a wrong reading of the question shows.
filters = list(
serious_only = serious_only,
term = term %||% "any",
body_system = body_system %||% "any"
),
subjects_with_event = n,
subjects_in_arm = total,
percent = round(100 * n / total, 1)
), auto_unbox = TRUE)
}
Four design choices make this function easier to inspect. It returns JSON because {ellmer} 0.5.0 deprecates returning data frames from tools. It includes the applied filters, exposing cases where the model misunderstood the request. It treats unknown terms as errors rather than silently converting them to zero, with a message that tells the model how to recover. And it checks every argument’s type: models may send values in a form the tool didn’t request, and later parts expose these functions to other programs through MCP. That type check came from a real testing problem. When `serious_only` arrived as the text `"true"` instead of the Boolean `TRUE`, the function quietly counted every subject with any adverse event: 217 instead of 3. Because this is an ordinary R function, you can also call it without a model. Source the R files first:
for (file in list.files("R", pattern = "[.]R$", full.names = TRUE)) source(file)
count_subjects(load_trial_data(), arm = "Xanomeline High Dose", term = "PRURITUS")
#> {"arm":"Xanomeline High Dose","filters":{"serious_only":false,"term":"PRURITUS","body_system":"any"},"subjects_with_event":25,"subjects_in_arm":72,"percent":34.7}
Here is the tool definition shown to the model:
ellmer::tool(
function(arm = NULL, serious_only = FALSE, term = NULL, body_system = NULL) {
count_subjects(data, arm, serious_only, term, body_system)
},
name = "count_subjects",
description = paste(
"Count subjects in the safety population with at least one treatment-emergent",
"adverse event matching the filters. Returns the count, the number of subjects",
"in the arm and the percentage."
),
arguments = list(
arm = arm_type,
serious_only = ellmer::type_boolean("Only count serious adverse events.", required = FALSE),
term = ellmer::type_string(
"MedDRA preferred term, e.g. 'PRURITUS'. Leave out to count any event.",
required = FALSE
),
body_system = ellmer::type_string(
"MedDRA body system, e.g. 'NERVOUS SYSTEM DISORDERS'. Leave out for any.",
required = FALSE
)
),
annotations = ellmer::tool_annotations(title = "Counting subjects", read_only_hint = TRUE)
)
The annotation supplies a title that part two’s app uses to label tool calls. The arm argument is an enum, so the model is given exactly three permitted values:
arm_type <- ellmer::type_enum(ARMS, "Treatment arm. Leave out to cover all arms.", required = FALSE)
`check_arm()` still validates the input in case a caller ignores that list. Altogether, `trial_tools()` builds four tools around one copy of the data: `list_arms()` returns group sizes, `ae_summary()` lists common adverse events, `count_subjects()` performs the filtered count, and `subject_profile()` retrieves one patient’s record.

Running the agent loop

A short function in `R/agent.R` creates a Gemini chat, supplies a system prompt described next, and registers the tools.

What the agent actually did

The useful result here is traceability: the model chose which counts to request, while R returned the figures it used. That division matters in a safety analysis, where a fluent answer is no substitute for a number you can trace back to a calculation.

The agent is created by reading a system prompt, opening a chat client, and registering the available tools:

make_agent <- function(tools = trial_tools(), prompt_path = "prompts/system.md",
model = NULL, chat_fun = ellmer::chat_google_gemini) {
chat <- chat_fun(system_prompt = read_prompt(prompt_path), model = model)
chat$register_tools(tools)
chat
}

In {ellmer} 0.5.0, leaving model as NULL selects gemini-3.7-flash. That default can change with a release, so part 4 names the model explicitly. Changing providers usually means changing one argument, for example make_agent(chat_fun = ellmer::chat_anthropic). Local models via chat_ollama() need an extra consideration, covered in part 4.

The demonstration asks two questions with echo = "all", which displays the question, tool calls, results, and final answer:

chat <- make_agent()
chat$chat("Which adverse events were most common on the high dose, compared with placebo?", echo = "all")
chat$chat("How many subjects had a serious adverse event, and on which arms?", echo = "all")

For the first question, the model requested the top ten events separately for the high dose and placebo. The high-dose results were:

{"arm":"Xanomeline High Dose","serious_only":false,"subjects_in_arm":72,"by":"term","rows":[{"event":"PRURITUS","subjects":25,"percent":34.7},{"event":"APPLICATION SITE PRURITUS","subjects":21,"percent":29.2},{"event":"APPLICATION SITE ERYTHEMA","subjects":14,"percent":19.4},{"event":"ERYTHEMA","subjects":14,"percent":19.4},{"event":"DIZZINESS","subjects":10,"percent":13.9},{"event":"APPLICATION SITE IRRITATION","subjects":9,"percent":12.5},{"event":"HYPERHIDROSIS","subjects":8,"percent":11.1},{"event":"RASH","subjects":8,"percent":11.1},{"event":"SINUS BRADYCARDIA","subjects":8,"percent":11.1},{"event":"APPLICATION SITE DERMATITIS","subjects":7,"percent":9.7}]}

Four of those ten events did not appear in the placebo top ten, so the model made additional count_subjects() calls for placebo: dizziness, application site irritation, hyperhidrosis, and sinus bradycardia. Its answer compared all ten events across the arms, whose safety populations were 72 on high dose and 86 on placebo:

  • Pruritus: 25 (34.7%) versus 8 (9.3%)
  • Application site pruritus: 21 (29.2%) versus 6 (7.0%)
  • Application site erythema: 14 (19.4%) versus 3 (3.5%)
  • Erythema: 14 (19.4%) versus 8 (9.3%)
  • Dizziness: 10 (13.9%) versus 2 (2.3%)
  • Application site irritation: 9 (12.5%) versus 3 (3.5%)
  • Hyperhidrosis: 8 (11.1%) versus 2 (2.3%)
  • Rash: 8 (11.1%) versus 5 (5.8%)
  • Sinus bradycardia: 8 (11.1%) versus 2 (2.3%)
  • Application site dermatitis: 7 (9.7%) versus 5 (5.8%)

For serious adverse events, it queried all arms and then each treatment arm with serious_only = TRUE. The tool results showed 3 of 254 subjects overall (1.2%): none of 86 on placebo, 2 of 96 on low dose (2.1%), and 1 of 72 on high dose (1.4%). The answer repeated those counts and denominators.

That’s a reasonably legible tool trail. The first answer used two ae_summary() calls plus four placebo counts; the second used four count_subjects() calls. Every figure in both answers appeared in a tool result. To run the demo, use source("demo/part-1.R"); it asks one question without tools and two with the agent, taking about six requests. Offline checks run with source("tests/run.R") and make no API calls.

The prompt helps, but it can’t enforce the rules

The system prompt defines the agent’s job and the limits on its answers:

You answer questions about the safety data of one clinical trial: CDISC pilot study 01, which compared a xanomeline skin patch at two doses with placebo.
- Use the tools for every fact about the trial. Every number in your answer must appear in a tool result from this conversation. Never calculate, estimate or remember numbers yourself.
- Quote numbers exactly as the tools give them.
- Say which arm each number is for. Counts are subjects in the safety population, by the treatment they actually received, with treatment-emergent adverse events.
- If the tools cannot answer a question, say so.
- Keep answers short.

Two instructions carry most of the weight: every trial fact must come from a tool, and every number must be quoted as returned. Requiring the treatment arm and population gives readers a denominator they can check. Still, a prompt is guidance, not a guarantee. Part 3 adds a check that verifies where each answer’s numbers came from.

A single run_r_code() tool might seem more flexible, but it would leave you checking whatever code the model wrote. Narrow tools are easier to test against hand-checked values, log, and permit or block individually. They also give the model a smaller task: choose a counting operation instead of writing correct {dplyr} code.

What still needs work

This is a working demonstration, not a team-ready system. The gaps are practical ones:

  • Numbers can still be invented or rounded. The prompt asks the model to quote tool output, but part 3 is where the series checks every number in an answer against those results.
  • Patient records need tighter access. The current agent can open any patient’s record. Part 3 restricts subject_profile() to a reviewer group, whose members confirm each record before it opens. Part 5 omits that tool from the set shared with other AI agents.
  • A console is a one-person interface. Part 2 puts the agent in a Shiny app, with a separate conversation for each user.
  • Free-tier limits and outages interrupt use. Within minutes, the team encountered “503 high demand” and “429 quota exceeded” errors. At the time of writing, the allowance was about 20 requests a day per model, and a question could consume one request per round of tool calls. {ellmer} retries each request up to three times by default; options(ellmer_max_tries = …) changes that setting. Parts 2 and 4 address the remaining issues.

The real test is control over the numbers

An agent combines a model, tools, and a loop: the model chooses what to ask, R counts against filtered data, and the prompt tells the model to report the returned values. In this example, the tool trail made the answers auditable. That matters because the two confident answers produced without tools disagreed with each other and with the data.

The next part moves the agent into a Shiny app so multiple people can use it at once. The code for this installment, including demo/part-1.R and the offline tests, is in the blog repository.

If you’re building something similar, the call for talks is open until 29 January 2027 for AI in Production 2027. For a guided introduction to LLM applications with R and Python, see the LLM-Driven Applications with R and Python course.

Updates and revisions are available in the original post.

To leave a comment for the author, follow the link to The Jumping Rivers Blog.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R, along with other topics. Post or find an R/data-science job.
Want to share your content on R-bloggers? Submit your blog, or use R-Posts.
Continue reading:Agents in Production: Build Your First AI Agent in R with {ellmer}
Source: The Jumping Rivers Blog · www.r-bloggers.com

Comments

Sign in to comment.
No comments yet. Be the first to comment.

Related Articles

Agents in Production: Build Your First AI Agent in R with...