Testing Third-Party Services with Minitest and VCR
In the previous article of this series we have set up Rails 5 application with Minitest framework and have written a model and a system test. Imagine, however, that our application also communicates with some third-party API and we would like to test this process as well. A handful of potential problems arise:
* API’s response time may be quite significant which means our tests will become much slower.
* API may be unavailable due to various reasons. The simplest case is when you are working from a location with no Internet access.
* API may have strict quota limitation which means that after running the same test over and over again your quote may be depleted.
* Data returned by the API may change therefore breaking your assertions. A trivial example would be an API that returns currency rates: each day the rates usually change and so you will need to tweak the assertions every time.
But luckily all these problems can be overcome with a solution called [VCR]. VCR is a Ruby gem that allows you to record and later playback HTTP interactions in your tests.
In this article you will learn how to work with VCR, how to configure it and how to write tests for third-party services. Let’s get started!
The source code for the MiniTest
Introducing a Third-Party API
Before writing any tests we of course need to setup communication between our application and some API. I don’t want this API to be very complex, so let’s, for example, take advantage of the OpenWeatherMap API that is quite simple and has a free version. Before proceeding, you will need to register at OpenWeatherMap and proceed to the API keys section. Here you will find a key that may be utilized to perform API requests. Please note that it may take about 10 minutes for this key to become active.
Now we need to store this key in a safe location. I’ll stick with a dotenv-rails gem that allows to easily load environment variables. The idea is that the API key should not be publically exposed therefore we cannot simply hard-code it in our project. So, drop in a new gem:
``` # Gemfile group :development, :test do # ... gem 'dotenv-rails' end ```
Run:
$ bundle install
Then create a new *.env* file in the project’s root with the following contents:
OPENWEATHER_KEY: YOUR_API_KEY
Lastly, exclude the *.env* file from version control by modifying *.gitignore*:
.env
Great! Now let’s code a very simple API wrapper to work with the OpenWeatherMap service:
ruby
app/services/weather_api.rb
class WeatherApi
attr_reader :data
def initialize(query)
@query = query
@data = get_and_parse request_uri
end
end
In this article we will only work with the Current weather data endpoint that shows the weather for a given location (based on the query).
Add two private methods to construct a proper URI and parse a response:
ruby
# ...
private
def get_and_parse(uri)
JSON.parse Net::HTTP.get(uri)
end
def request_uri
URI.parse "https://api.openweathermap.org/data/2.5/weather?q=#{@query}&appid=#{ENV['OPENWEATHER_KEY']}&units=metric"
end
Now add two interface methods to fetch temperature (in Celsius) and weather conditions (like rain, for example):
ruby
# ...
def initialize
# ...
end
def temperature
@data['main']['temp']
end
def weather_conditions
@data['weather']
end
Nice, but we can do a bit better. Data returned by the API can be cached because, after all, weather conditions does not usually change drastically every 5 minutes. So, let’s utilize low-level Rails caching:
ruby
# ...
def initialize(query)
@query = query
@data = Rails.cache.fetch("weather_api/#{@query}", expires_in: 12.hours) do
get_and_parse request_uri
end
end
Now every separate query will be cached for 12 hours (of course, you may adjust this value as needed). Note, however, that by default Rails will not perform caching in development environment. Let’s take a look at *config/environments/development.rb* file:
ruby
# ...
if Rails.root.join('tmp/caching-dev.txt').exist?
config.action_controller.perform_caching = true
config.cache_store = :memory_store
config.public_file_server.headers = {
'Cache-Control' => "public, max-age=#{2.days.seconds.to_i}"
}
else
config.action_controller.perform_caching = false
config.cache_store = :null_store
end
This piece of code checks if a *tmp/caching-dev.txt* file exists and enables caching if yes. So, you may simply create this file manually or by using command.
$ rails dev:cache
Displaying Weather Data
Now that we have our weather API wrapper in place, let’s utilize it by displaying a short information about weather conditions in some location. Of course, ideally the user should enter his location upon signing up, but we don’t have this feature yet. Therefore, let’s hardcode the query instead:
# app/controllers/posts_controller.rb
# ...
def index
@weather = WeatherApi.new 'Moscow,ru'
@posts = Post.all
end
Of course, you may change the query as you like. Now display the returned data:
```html
<!-- app/views/posts/index.html.erb -->
<aside id="weather-status">
Temperature is <%= @weather.temperature %><sup>o</sup>C
<% if @weather.weather_conditions.any? %>
<p>Conditions:</p>
<div>
<% @weather.weather_conditions.each do |w| %>
<%= "#{w['main']} (#{w['description']})" %><br>
<% end %>
</div>
<% end %>
</aside>
<!-- ... -->
```
weather_conditions may return an array, therefore we firstly need to check if it has any values at all. Of course, you may introduce an additional level of complexity and load the weather data asynchronously, but it is not needed for the purposes of this article.
Great, now we have something to test and so let’s proceed to integrating VCR!
Integrating VCR
First of all, we need to add two gems in the *Gemfile*: VCR itself and some library to perform stubbing. I’ll go with Webmock but there are other solutions available.
```ruby # Gemfile group :test do gem 'simplecov', require: false gem 'vcr' gem 'webmock' end ```
Run:
$ bundle install
Next create a VCR configuration file:
```ruby
# test/support/vcr.rb
require 'vcr'
VCR.configure do |config|
config.cassette_library_dir = Rails.root.join 'test', 'vcr_cassettes'
config.hook_into :webmock
config.ignore_localhost = true
config.allow_http_connections_when_no_cassette = false
config.filter_sensitive_data("{appid}"){ ENV.fetch("OPENWEATHER_KEY") }
end
```
* cassette_library_dir explains where the recorded interactions should reside. You may choose a different location, but do not place these files into the *fixtures* directory as Rails will treat them as model fixtures. “Cassettes” is a fancy name for the files generated by VCR. They are named so because we are literally recording the interaction so that it can be played back later.
* hook_into says that we’d like to utilize Webmock.
* ignore_localhost disables VCR for local interactions.
* allow_http_connections_when_no_cassette is quite self-explaining. If a request to a third-party service is performed without any cassette specified, the test fails.
* filter_sensitive_data explains that the API key should not be saved on the cassettes. It will be replaced with the `{appid}` string.
There are other options available and you may find them at the VCR official docs.
Now let’s instruct Minitest to load all files and folders from the *support* directory:
```ruby
# test/test_helper.rb
# ...
Dir[Rails.root.join("test/support/**/*.rb")].each { |f| require f }
```
Lastly, create a *vcr_cassettes* folder.
This is pretty much it: VCR is now integrated and we can start using it!
Testing the API Wrapper
So, let’s see VCR in action by testing our WeatherApi wrapper. Create a new *test/services/weather_api_test.rb* file:
```ruby require 'test_helper' class WeatherApiTest < ActiveSupport::TestCase end ```
Inside I’d like to perform an API request, record it and then set some assertions. The first two steps can be done inside a setup method:
```ruby
# ...
class WeatherApiTest < ActiveSupport::TestCase
def setup
VCR.use_cassette("moscow weather") do
@weather = WeatherApi.new 'Moscow,ru'
end
end
end
```
The idea is really simple. We are wrapping an API interaction with a block passed to the `use_cassette` method. This method works in the following way:
* If the specified cassette (*moscow_weather.yml* in this case) cannot be found, a real API request is performed. The reponse is saved to the YAML file.
* If the cassette is found, the recorded data is used right away and no request is being performed. This behaviour can be changed by setting a different record mode.
Once again note that if you don’t wrap the API interaction with the use_cassette method, the test will fail right away because we have set the allow_http_connections_when_no_cassette option to false.
Now just set some assertions:
```ruby
# ...
test 'it should return temperature' do
assert_equal 8, @weather.temperature
end
test 'it should return weather conditions' do
weather_condition = @weather.weather_conditions.first
assert_equal 'Rain', weather_condition['main']
assert_equal 'light intensity shower rain', weather_condition['description']
end
```
Run the tests:
$ rails test
Most likely the assertions will fail, because the returned temperature and weather conditions will be different to what I have specified in the listing above.
Note that the tests take quite some time to complete which means the API request is performed. Now the *vcr_cassette* folder should contain a *moscow_weather.yml* file that stores the recorded response. You may open it and find the body section that looks like this:
```yaml
body:
encoding: UTF-8
string: '{"coord":{"lon":37.62,"lat":55.75},"weather":[{"id":520,"main":"Rain","description":"light
intensity shower rain","icon":"09n"}] ........
```
That’s the response that we’ve got from the API. You may now set your assertions according to this response and run the test again. It should be finished much faster than the previous time because in this case we are using the recorded data. How cool is that?
Writing a System Test
Before wrapping up, let’s also write a small system test to make sure that the weather condition is being displayed properly on the root page. We already have the *test/system/posts_test.rb* file, so let’s add yet another test example inside:
```ruby
# ...
test 'all posts page should also display weather information' do
VCR.use_cassette("moscow weather") do
end
end
```
I am using the VCR cassette here because the query is pretty much the same. Without the cassette the test will fail as previously explained. Now just flesh out the test like this:
```ruby
# ...
test 'all posts page should also display weather information' do
VCR.use_cassette("moscow weather") do
visit posts_url
within '#weather-status' do
assert_text 'Temperature is 8'
end
end
end
```
within is a method provided by Capybara that scopes the interactions to the given element. After all, we are only interested in the #weather-status block in this test.
Run your system tests:
$ rails test:system
And it should succeed without any problems which means that our API interaction works perfectly!
Conclusion: –
In this article we have discussed how to test third-party services with the help of VCR. You have learned how to integrate this gem into the Rails application, how to configure it and how to work with cassettes in order to record API interactions. As you see, there is absolutely nothing complex about this gem and it can really make your life simpler!
If you would like to learn more about Test-Driven Development and using Minitest with Rails, I recommend referring to our course “Beginners Guide to Test Driven Development” which spans more than 4 hours and provides lots of useful information as well as practical examples.
I thank you for staying with me and see you soon!
