ellmer 0.2.0

I’m thrilled to announce the release of ellmer 0.2.0! ellmer is an R package designed to make it easy to use large language models (LLMs) from R. It supports a wide variety of providers (including OpenAI, Anthropic, Azure, Google, Snowflake, Databricks and many more), makes it easy to extract structured data, and to give the LLM the ability to call R functions via tool calling.

You can install it from CRAN with:

install.packages("ellmer")

Before diving into the details of what’s new, I wanted to welcome Garrick Aden-Buie to the development team! Garrick is one of my colleagues at Posit, and has been instrumental in building out the developer side of ellmer, particularly as it pertains to tool calling and async, with the goal of making shinychat as useful as possible.

In this post, I’ll walk you through the key changes in this release: a couple of breaking changes, new batched and parallel processing capabilities, a cleaner way to set model parameters, built-in cost estimates, and general updates to our provider ecosystem. This was a giant release, and I’m only touching on the most important topics here, so if you want all the details, please check out the release notes.

library(ellmer)

Breaking changes

Before we dive into the cool new features, we need to talk about the less fun stuff: some breaking changes. As the ellmer package is still experimental (i.e. it has not yet reached 1.0.0), we will be making some breaking changes from time-to-time. That said, we’ll always provide a way to revert to the old behaviour and will generally avoid changes that we expect will affect a lot of existing code. There are three breaking changes in this release:

If you save a Chat object to disk, the API key is no longer recorded. This protects you from accidentally saving your API key in an insecure location at the cost of not allowing you to resume a chat you saved to disk (we’ll see if we can fix that problem in the future).
We’ve made some refinements to how ellmer converts JSON to R data structures. The most important change is that tools are now invoked with their inputs converted to standard R data structures. This means you’ll get proper R vectors, lists, and data frames instead of raw JSON objects, making your functions easier to write. If you prefer the old behavior, you can opt out with tool(convert = FALSE).
The turn argument has been removed from the chat_ functions; use Chat$set_turns() instead.
Chat$tokens() has been renamed to Chat$get_tokens() and it now returns a correctly structured data frame with rows aligned to turns.

Batch and parallel chat

One of the most exciting additions in 0.2.0 is support for processing multiple chats efficiently. If you’ve ever found yourself wanting to run the same prompt against hundreds or thousands of different inputs, you now have two powerful options: parallel_chat() and batch_chat().

parallel_chat() works with any provider and lets you submit multiple chats simultaneously:

chat <- chat_openai()
#> Using model = "gpt-4.1".
prompts <- interpolate("
  What do people from {{state.name}} bring to a potluck dinner?
  Give me the top three things.
")

results <- parallel_chat(chat, prompts)
# [working] (32 + 0) -> 10 -> 8 | ■■■■■■                            16%

This doesn’t save you money, but it can be dramatically faster than processing chats sequentially. (Also note that interpolate() is now vectorised, making it much easier to generate many prompts from vectors or data frames.)

batch_chat() currently works with OpenAI and Anthropic, offering a different trade-off:

chat <- chat_openai()
#> Using model = "gpt-4.1".
results <- batch_chat(chat, prompts, path = "potluck.json")
results[[1]]
#> <Chat OpenAI/gpt-4.1 turns=2 tokens=26/133 $0.00>
#> ── user [26] ──────────────────────────────────────────────────────────────────────────────────────
#> What do people from Alabama bring to a potluck dinner?
#> Give me the top three things.
#> ── assistant [133] ────────────────────────────────────────────────────────────────────────────────
#> At a potluck dinner in Alabama, you'll most often find these top three dishes brought by guests:
#> 
#> 1. **Fried Chicken** – Always a southern staple, crispy homemade (or sometimes store-bought!) fried chicken is practically expected.
#> 2. **Deviled Eggs** – Easy to make, transport, and always a crowd-pleaser at southern gatherings.
#> 3. **Homemade Casserole** – Usually something like broccoli cheese casserole, hashbrown casserole, or chicken and rice casserole, casseroles are a potluck favorite because they serve many and are comforting.
#> 
#> Honorable mentions: banana pudding, macaroni and cheese, and cornbread.

Batch requests can take up to 24 hours to complete (although often finish much faster), but cost 50% less than regular requests. This makes them perfect for large-scale analysis where you can afford to wait. Since they can take a long time to complete, batch_chat() requires a path, which is used to store information about the state of the job, ensuring that you never lose any work. If you want to keep using your R session, you can either set wait = FALSE or simply interrupt the waiting process, then later, either call batch_chat() to resume where you left off or call batch_chat_completed() to see if the results are ready to retrieve. batch_chat() will store the chat responses in this file, so you can either keep it around to cache the results, or delete it to free up disk space.

Both functions come with structured data variations: batch_chat_structured() and parallel_chat_structured(), which make it easy to extract structured data from multiple strings.

prompts <- list(
  "I go by Alex. 42 years on this planet and counting.",
  "Pleased to meet you! I'm Jamal, age 27.",
  "They call me Li Wei. Nineteen years young.",
  "Fatima here. Just celebrated my 35th birthday last week.",
  "The name's Robert - 51 years old and proud of it.",
  "Kwame here - just hit the big 5-0 this year."
)
type_person <- type_object(name = type_string(), age = type_number())

data <- batch_chat_structured(
  chat = chat,
  prompts = prompts,
  path = "people-data.json",
  type = type_person
)
data
#>     name age
#> 1   Alex  42
#> 2  Jamal  27
#> 3 Li Wei  19
#> 4 Fatima  35
#> 5 Robert  51
#> 6  Kwame  50

This family of functions is experimental because I’m still refining the user interface, particularly around error handling. I’d love to hear your feedback!

Parameters

Previously, setting model parameters like temperature and seed required knowing the details of each provider’s API. The new params() function provides a consistent interface across providers:

chat1 <- chat_openai(params = params(temperature = 0.7, seed = 42))
#> Using model = "gpt-4.1".
chat2 <- chat_anthropic(params = params(temperature = 0.7, max_tokens = 100))
#> Using model = "claude-3-7-sonnet-latest".

ellmer automatically maps these to the appropriate provider-specific parameter names. If a provider doesn’t support a particular parameter, it will generate a warning, not an error. This allows you to write provider-agnostic code without worrying about compatibility.

params() is currently supported by chat_anthropic(), chat_azure(), chat_openai(), and chat_gemini(); feel free to file an issue if you’d like us to add support for another provider.

Cost estimates

Understanding the cost of your LLM usage is crucial, especially when working at scale. ellmer now tracks and displays cost estimates. For example, when you print a Chat object, you’ll see estimated costs alongside token usage:

chat <- chat_openai(echo = FALSE)
#> Using model = "gpt-4.1".
joke <- chat$chat("Tell me a joke")
chat
#> <Chat OpenAI/gpt-4.1 turns=2 tokens=11/20 $0.00>
#> ── user [11] ──────────────────────────────────────────────────────────────────────────────────────
#> Tell me a joke
#> ── assistant [20] ─────────────────────────────────────────────────────────────────────────────────
#> Why did the golfer bring two pairs of pants?  
#> In case he got a hole in one!

You can also access costs programmatically with Chat$get_cost() and see detailed breakdowns with tokens_usage():

chat$get_cost()
#> [1] $0.00

token_usage()
#>   provider   model input output price
#> 1   OpenAI gpt-4.1  1788   8952 $0.08

(The numbers will be more interesting for real use cases.)

Keep in mind that these are estimates based on published pricing. LLM providers make it surprisingly difficult to determine exact costs, so treat these as helpful approximations rather than precise accounting.

Provider updates

The ellmer ecosystem continues to grow! We’ve added support for three new providers:

Hugging Face via chat_huggingface(), thanks to Simon Spavound.
Mistral AI via chat_mistral().
Portkey via chat_portkey(), thanks to Maciej Banaś.

chat_snowflake() and chat_databricks() are now considerably more featureful, thanks to improvements in the underlying APIs. They now also both default to Claude Sonnet 3.7, and chat_databricks() picks up Databricks workspace URLs set in the Databricks configuration file, improving compatibility with the Databricks CLI.

We’ve also cleaned up the naming scheme for existing providers. The old function names still work but are deprecated:

chat_anthropic() replaces chat_claude().
chat_azure_openai() replaces chat_azure().
chat_aws_bedrock() replaces chat_bedrock().
chat_google_gemini() replaces chat_gemini().

And updated some default models: chat_anthropic() now uses Claude Sonnet 4, and chat_openai() uses GPT-4.1.

Finally, we’ve added a family of models_*() functions that let you discover available models for each provider:

tibble::as_tibble(models_anthropic())
#> # A tibble: 11 × 6
#>    id                        name  created_at          cached_input input output
#>    <chr>                     <chr> <dttm>                     <dbl> <dbl>  <dbl>
#>  1 claude-opus-4-20250514    Clau… 2025-05-22 00:00:00        NA    NA     NA   
#>  2 claude-sonnet-4-20250514  Clau… 2025-05-22 00:00:00        NA    NA     NA   
#>  3 claude-3-7-sonnet-202502… Clau… 2025-02-24 00:00:00         0.3   3     15   
#>  4 claude-3-5-sonnet-202410… Clau… 2024-10-22 00:00:00         0.3   3     15   
#>  5 claude-3-5-haiku-20241022 Clau… 2024-10-22 00:00:00         0.08  0.8    4   
#>  6 claude-3-5-sonnet-202406… Clau… 2024-06-20 00:00:00         0.3   3     15   
#>  7 claude-3-haiku-20240307   Clau… 2024-03-07 00:00:00         0.03  0.25   1.25
#>  8 claude-3-opus-20240229    Clau… 2024-02-29 00:00:00         1.5  15     75   
#>  9 claude-3-sonnet-20240229  Clau… 2024-02-29 00:00:00        NA    NA     NA   
#> 10 claude-2.1                Clau… 2023-11-21 00:00:00        NA    NA     NA   
#> 11 claude-2.0                Clau… 2023-07-11 00:00:00        NA    NA     NA

These return data frames with model IDs, pricing information (where available), and other provider-specific metadata.

Developer tools

This release includes several improvements for developers building more sophisticated LLM applications, particularly around tool usage and debugging.

The most immediately useful addition is echo = "output" in Chat$chat(). When you’re working with tools, this shows you exactly what’s happening as tool requests and results flow back and forth. For example:

chat <- chat_anthropic(echo = "output")
#> Using model = "claude-3-7-sonnet-latest".
chat$set_tools(btw::btw_tools("session"))
chat$chat("Do I have bslib installed?")
#> I can check if the 'bslib' package is installed in your R environment. Let me do that for you.
#> ◯ [tool call] btw_tool_session_check_package_installed(package_name = "bslib", intent = "Checking
#> if bslib package is installed")
#> ● #> Package `bslib` version 0.9.0 is installed.
#> Yes, you have the bslib package installed. It's version 0.9.0 on your system.
#> 
#> The bslib package is a Bootstrap utility package for R that helps create modern web interfaces in 
#> Shiny apps and R Markdown documents. It provides tools for customizing Bootstrap themes, creating 
#> page layouts, and building interactive card components.

For more advanced use cases, we’ve added tool annotations via tool_annotations(). These follow the Model Context Protocol and let you provide richer descriptions of your tools:

weather_tool <- tool(
  fun = get_weather,
  description = "Get current weather for a location",
  .annotations = tool_annotations(
    audience = list("user", "assistant"),
    level = "beginner"
  )
)

We’ve also introduced tool_reject(), which lets you reject tool requests with an explanation:

my_tool <- tool(function(dangerous_action) {
  if (dangerous_action == "delete_everything") {
    tool_reject("I can't perform destructive actions")
  }
  # ... normal tool logic
})

Acknowledgements

A big thanks to all 67 contributors who helped out with ellmer development through thoughtful discussions, bug reports, and pull requests. @13479776, @adrbmdns, @AlvaroNovillo, @andersolarsson, @andrie, @arnavchauhan7, @arunrajes, @asb2111, @atheriel, @bakaburg1, @billsanto, @bzzzwa, @calderonsamuel, @christophscheuch, @conorotompkins, @CorradoLanera, @david-diviny-nousgroup, @DavisVaughan, @dm807cam, @dylanpieper, @edgararuiz, @gadenbuie, @genesis-gh-yshteyman, @hadley, @Ifeanyi55, @jcheng5, @jimbrig, @jsowder, @jvroberts, @kbenoit, @kieran-mace, @kleinlennart, @larry77, @lindbrook, @maciekbanas, @mark-andrews, @Marwolaeth, @mattschaelling, @maurolepore, @michael-dewar, @michaelgrund, @mladencucak, @mladencucakSYN, @moodymudskipper, @mrembert, @natashanath, @noslouch, @pedrobtz, @prasven, @ries9112, @s-spavound, @schloerke, @schmidb, @scjohannes, @seawavevan, @simonpcouch, @smach, @sree1658, @stefanlinner, @szzhou4, @t-kalinowski, @trafficfan, @Vinnish-A, @vorpalvorpal, @walkerke, @wch, and @WickM.