httr2 1.2.0
We’re delighted to announce the release of httr2 1.2.0! httr2 (pronounced “hitter2”) is a comprehensive HTTP client that provides a modern, pipeable API for working with web APIs. It builds on top of {curl} to provide features like explicit request objects, built-in rate limiting & retry tooling, comprehensive OAuth support, and secure handling of secrets and credentials.
You can install it from CRAN with:
install.packages("httr2")
This blog post will walk you through the most important changes in 1.2.0: lifecycle updates, improved security for redacted headers, URL handlimg improvements, improved debugging tools, and a handful of other quality of life improvements. You can see a full list of changes in the release notes
Lifecycle changes
Part of httr2’s continued evolution is phasing out features that we now believe were mistakes. In this release:
-
req_perform_stream()
has been soft deprecated (not just superseded) in favour ofreq_perform_connection()
. -
Deprecated functions
multi_req_perform()
,req_stream()
,with_mock()
, andlocal_mock()
have now been removed. Please use their modern replacements,req_perform_parallel()
,req_perform_stream()
,with_mocked_responses()
, andlocal_mocked_responses()
, instead. -
Deprecated arguments
req_perform_parallel(pool)
,req_oauth_auth_code(host_name, host_ip, port)
, andoauth_flow_auth_code(host_name, host_ip, port)
have been removed. Please usereq_perform_parallel(max_active)
andreq_oauth_auth_code(redirect_url)
/oauth_flow_auth_code(redirect_url)
instead.
Enhanced security for redacted headers
One of the most important improvements in this release improves the security of redacted headers. Redacted headers are used to conceal secrets, like API keys or passwords, that you don’t want to accidentally reveal. For a long time, httr2 has automatically hidden these headers when you
print()
or
str()
them, ensuring that they don’t accidentally end up in log files. You can see this in action with the Authorization
header, which httr2 now automatically redacts:
req <- request("http://example.com") |>
req_auth_basic("username", "password")
req
#> <httr2_request>
#> GET http://example.com
#> Headers:
#> * Authorization: <REDACTED>
#> Body: empty
str(req$headers)
#> <httr2_headers>
#> $ Authorization: <REDACTED>
req |> req_dry_run()
#> GET / HTTP/1.1
#> accept: */*
#> accept-encoding: deflate, gzip
#> authorization: <REDACTED>
#> host: example.com
#> user-agent: httr2/1.2.0 r-curl/6.4.0 libcurl/8.14.1
(If for you do really need to see the redacted values you can get with a bit of extra effort: call the new
req_get_headers()
function with redacted = "reveal"
.)
In httr2 1.2.0, we’ve gone one step further, and prevented redacted headers from being saved to disk. Now if you save and reload a request, you’ll notice that the redacted headers are no longer present:
path <- tempfile()
saveRDS(req, path)
req2 <- readRDS(path)
req2 |> req_dry_run()
#> GET / HTTP/1.1
#> accept: */*
#> accept-encoding: deflate, gzip
#> host: example.com
#> user-agent: httr2/1.2.0 r-curl/6.4.0 libcurl/8.14.1
This protects you from accidentally revealing your credentials if you save a request to disk. This is easier to do than you might expect because httr2 includes the request object in every response (since this makes debugging much easier). That means if you’re caching a slow response, it’s very easy to accidentally store a secret, potentially leaking secure values. (Don’t ask me how I discovdred this!)
URL handling improvements
URL construction is now powered by
curl::curl_modify_url()
, which correctly escapes the path component:
req <- request("https://api.example.com")
req |> req_url_path("/users/john doe/profile") |> req_get_url()
#> [1] "https://api.example.com/users/john%20doe/profile"
This means that
req_url_path()
can now only affect the path component of the URL, not the query parameters. If you previously relied on this behaviour, you’ll need to switch to
req_url_query()
:
# won't work any more:
req |>
req_url_path("/users?name=john-doe") |>
req_get_url()
#> [1] "https://api.example.com/users%3Fname%3Djohn-doe"
# so now do this:
req |>
req_url_path("/users") |>
req_url_query(name = "john-doe") |>
req_get_url()
#> [1] "https://api.example.com/users?name=john-doe"
Improved debugging tools
The vast majority of modern APIs use JSON, so httr2 now includes a few features to make debugging those APIs a little easier:
-
last_request()
andlast_response()
are now paired withlast_request_json()
andlast_response_json()
which pretty-print the JSON bodies of the last request and response. -
req_dry_run()
andreq_verbose()
automatically pretty print JSON bodies (turn this off by settingoptions(httr2_pretty_json = FALSE)
).
We’ve also included a few general tools to make it easier to control httr2’s default verbosity. You can now control the default via the HTTR2_VERBOSITY
environment variable and there’s a new
local_verbosity()
function to match the existing
with_verbosity()
.
Quality of life improvements
This release also includes a bunch of few smaller quality of life improvements:
-
req_perform_parallel()
now lifts many of its restrictions. It now supports simplified versions ofreq_throttle()
andreq_retry()
, it can refresh OAuth tokens, and it checks the cache before each request. -
req_get_url()
,req_get_method()
,req_get_headers()
,req_body_get_type()
, andreq_get_body()
allow you to introspect request objects. -
req_throttle()
now uses a “ leaky bucket". This maintains the same average rate limit as before, while allowing bursts of higher rates. -
resp_timing()
exposes detailed timing information for a response.
Acknowledgements
A big thanks to everyone who contributed to this release through issues, pull requests, and discussions on GitHub: @Aariq, @annalena13, @apsteinmetz, @arcresu, @arnaudgallou, @atheriel, @DavidRLovell, @dfalbel, @Eli-Berkow, @fwimp, @hadley, @jansim, @jcheng5, @jeffreyzuber, @jjesusfilho, @jonthegeek, @Kevanness, @m-muecke, @maelle, @mayeulk, @mdsumner, @noamross, @omuelle, @pedrobtz, @plietar, @ramiromagno, @salim-b, @sckott, @shikokuchuo, @vibalre, @vladimirobucina, and @ZheFrench.