#rails

TILSeptember 17, 2019by Alexander Budchanov

Renaming keys in PostgreSQL JSON / Migration template for gem Recorder

In our project, we used gem Recorder. It saves changes in jsonb format.

Once we need to rename a column in model Contact, but we didn't want to lose logs.

I introduce you template of migration for this case.


class RenameOldColumnToNewColumn < ActiveRecord::Migration[5.2]

  def up

    rename_column :contacts, :old_column, :new_column



    execute <<~SQL

      UPDATE recorder_revisions SET data = jsonb_set(data #- '{changes,old_column}', '{changes,new_column}', data#>'{changes,old_column}') WHERE data#>'{changes}'?'old_column';

    SQL

  end



  def down

    rename_column :contacts, :new_column, :old_column



    execute <<~SQL

      UPDATE recorder_revisions SET data = jsonb_set(data #- '{changes,new_column}', '{changes,old_column}', data#>'{changes,new_column}') WHERE data#>'{changes}'?'new_column';

    SQL

  end

end

You can simple replace contacts, new_column and old_column to your table name and columns names.

TILSeptember 18, 2019by Andrey Morozov

Strip Whitespace from Heredocs in Ruby/Rails

UPDATE!

Simple way - it's use:


class Template

  def self.base

    <<~TEXT

      Lorem Ipsum is simply dummy

      Lorem Ipsum is simply dummy

    TEXT

  end

end



> Template.base

=> "Lorem Ipsum is simply dummy\nLorem Ipsum is simply dummy\n"

I forgot about <<~, and that's why I got this TIL bike.


You use heredoc in Ruby/Rails app?

Then you're familiar with the problem..


class Template

  def self.base

    <<-TEXT

      Lorem Ipsum is simply dummy

      Lorem Ipsum is simply dummy

      Lorem Ipsum is simply dummy

    TEXT

  end

end



> Template.base

=> "      Lorem Ipsum is simply dummy\n      Lorem Ipsum is simply dummy\n      Lorem Ipsum is simply dummy\n"

hmm...

I don't want to have so many gaps in the text.

Maybe use them:


class Template

  def self.base

    <<-TEXT

Lorem Ipsum is simply dummy

Lorem Ipsum is simply dummy

    TEXT

  end

end



> Template.base

=> "Lorem Ipsum is simply dummy\nLorem Ipsum is simply dummy\n"

It's good, but the code looks like 💩.

This problem can be solved by using activesupport #strip_heredoc:


class Template

  def self.base

    <<-TEXT.strip_heredoc

      Lorem Ipsum is simply dummy

      Lorem Ipsum is simply dummy

    TEXT

  end

end



> Template.base

=> "Lorem Ipsum is simply dummy\nLorem Ipsum is simply dummy\n"

If you don't have active_support, use refine String with #strip_heredoc which will give similar result:


module Utils 

  refine String do 

    def strip_heredoc

      gsub(/^#{scan(/^[ \t]*(?=\S)/).min}/, "".freeze)

    end

  end

end



class Template

  using Utils

  def self.base

    <<-TEXT.strip_heredoc

      Lorem Ipsum is simply dummy

      Lorem Ipsum is simply dummy

    TEXT

  end

end



> Template.base 

=> "Lorem Ipsum is simply dummy\nLorem Ipsum is simply dummy\n"

TILSeptember 17, 2019by Alexander Budchanov

Rails 5.2 changes in callbacks

In version 5.1 you may see deprecation warnings in after_save callbacks (related to changes in ActiveRecord::Dirty module).

But since 5.2 these changes were applied.

For examples, I will use Rails 4.2.11 and Rails 5.2.3 and model User with email attribute. Let's do:


u = User.new(email: 'old@domain.com')

u.save

u.email = 'new@domain.com'

u.save

and look at after_save callback in time of last save.

1. attribute_changed?

Rails 4


> email_changed?

=> true

Rails 5.2


> email_changed?

=> false

but you can use saved_changes?


> saved_change_to_email?

=> true

2. changed?

Rails 4


> changed?

=> true

Rails 5.2


> changed?

=> false

but you can use saved_changes?


> saved_changes?

=> true

3. changes

Rails 4


> changes

=> {"email"=>["old@domain.com", "new@domain.com"]}

Rails 5.2


> changes

=> {}

but you can use saved_changes


> saved_changes

=> {"email"=>["old@domain.com", "new@domain.com"]}

4. previous_changes

Rails 4


> previous_changes

=> {"email"=>[nil, "old@domain.com"]}

Rails 5.2

Now, this method returns the changes that were just saved (like saved_changes).


> previous_changes

=> {"email"=>["old@domain.com", "new@domain.com"]}

this method has no replacement.

TILSeptember 13, 2019by Dmitry Voronov

How to use Rails translations for Reform attributes

If you use Reform for the form objects and want the translations in your Rails application to work as with ActiveRecord objects, then you can add ActiveModel::Translation module to the form class or base class and specify the method with the key to translations.


class BaseForm < Reform::Form

  extend ActiveModel::Translation



  def self.i18n_scope

    :forms

  end

end


class UserForm < BaseForm

  feature Coercion



  property :login

  property :change_password, virtual: true, type: Types::Form::Bool

  property :password

  property :password_confirmation



  validation do

    required(:login).filled

  end



  validation if: ->(_) { change_password } do

    required(:password).confirmation

  end

end

Set translations for form fields.

You can move them to a new file config/locales/forms.yml.


en:

  forms:

    attributes:

      user_form:

        login: Enter login

        change_password: Change password?

        password: Enter password

        password_confirmation: Confirm password

Now, translations will be used by the simple form or you can call them manually by UserForm.human_attribute_name.

TILAugust 30, 2019by Alexander Spitsyn

Testing external API integration with VCR gem

Suppose your application connects to an external service via API and you have a wrapper for this API that handles and parses response. The VCR gem gives you ability to store parsed response in a special format (cassetes). VCR makes a real request to the API for the first test run and writes it's response to the cassete for next test runnings.

First you need to set VCR configuration:


VCR.configure do |config|

  config.cassette_library_dir = "spec/vcr_cassettes"

  config.hook_into :webmock

end

And then write specs like the following:


RSpec.describe ExternalService do

  describe '#new_order' do

    let(:params) { { foo: 'bar' } }



    it 'creates new order' do

      VCR.use_cassette("external_service") do

        result = subject.new_order(params) # makes HTTP POST to an external service



        expect(result.successful?).to be_truthy

        expect(result.data).to have_key('order_id')

      end

    end

  end

end

See more details on the official page of the gem.

TILAugust 28, 2019by Ilia Kriachkov

Ruby double splat (**) operator cheatsheet

The operator ** is useful as an options hash.


def one_method(**options);end

This form is completely similar to the following:


def another_method(options = {});end

In addition, you can strictly define the set of required keys for the method.


def one_strict_method(first_name:, last_name: , **options)

  puts "options: #{options}"

  greeting = "Hello #{first_name} #{last_name}"

  puts options[:upcase] ? greeting.upcase : greeting

end


pry(main)> one_strict_method(upcase: true)

ArgumentError: missing keywords: first_name, last_name



pry(main)> one_strict_method(first_name: 'John', last_name: 'Doe', upcase: true)

options: {:upcase=>true}

HELLO JOHN DOE

=> nil

Another advantage of double splat literal is that it works like #merge for Ruby Hash


class Contact::ShowRepresenter #:nodoc:

  def call(contact)

    {

      contact: {

        **base_info(contact),

        **legal_info(contact),

        # You can add something more complex here.

        # **GeoLocaionRepresenter.new.(contact)

      }

    }

  end



  private



  def base_info(contact)

    {

      id: contact.id,

      first_name: contact.first_name,

      last_name: contact.last_name,

      email: contact.email,

      phone: contact.phone

    }

  end



  def legal_info(contact)

    {

      legal_name: contact.legal_name,

      legal_type: contact.legal_type,

      mailing_address: contact.mailing_address

    }

  end

end


pry(main)>Contact::ShowRepresenter.new.(Contact.last)

=> {

  :contact=>{

    :id=>51986,

    :first_name=>"Ilia",

    :last_name=>"Kriachkov",

    :email=>"ilia.kriachkov@jetrockets.ru",

    :phone=>"+79000000000",

    :legal_name=>"JetRockets",

    :legal_type=>"LLC",

    :mailing_address=>"15 Let Oktyabrya Street, #10b, Tver, Russian Federation 170008"

  }

}

In conclusion, I want to demonstrate some benchmark results.

As you can see, the ** operator is a bit faster than Hash#merge.


require 'benchmark'

n = 50_000

Benchmark.bm(2) do |x|

  x.report('merge:             ') { n.times { merge } }

  x.report('double_splat_merge:') { n.times { double_splat_merge } }

end



def merge

  hash = { a: 'a' }

  { b: 'b' }.merge(hash)

end



def double_splat_merge

  hash = { a: 'a' }

  { b: 'b', **hash }

end


                     user      system      total        real

merge:               0.109247   0.088652   0.197899 (  0.204470)

double_splat_merge:  0.079480   0.003590   0.083070 (  0.083642)

TILAugust 26, 2019by Dmitry Voronov

How to store large JSON in PostgreSQL with Rails Attributes API

If you store large objects in the database (such as JSON), for example, data for big reports, then this can take up a lot of space. To reduce the size of data, you can compress and store in binary form.

PostgreSQL has a bytea field type for storing such data. You can add bytea column in Rails using migration


add_column :reports, :data, :binary

For binary field operations, you can use the Rails Attributes API and add a new BinaryHash data type


# app/types/binary_hash.rb



class BinaryHash < ActiveRecord::Type::Binary

  def serialize(value)

    super value_to_binary(value.to_json)

  end



  def deserialize(value)

    super case value

          when NilClass

            {}

          when ActiveModel::Type::Binary::Data

            value_to_hash(value.to_s)

          else

            value_to_hash(PG::Connection.unescape_bytea(value))

          end

  end



  private



  def value_to_hash(value)

    JSON.parse(

      ActiveSupport::Gzip.decompress(value),

      symbolize_names: true

    ) || {}

  end



  def value_to_binary(value)

    ActiveSupport::Gzip.compress(value)

  end

end

Register new type in initializers


# config/initializers/types.rb



ActiveRecord::Type.register(:binary_hash, BinaryHash)

And add to binary type attribute in model


# app/models/snapshot.rb



class Reports < ApplicationRecord

  attribute :data, :binary_hash

end

Tests show that data size is reduced by almost 3 times


Run time with 100000 width JSON

                           user     system      total        real

Compress JSON          0.008671   0.001535   0.010206 (  0.010885)

Decompress JSON        0.001357   0.000095   0.001452 (  0.001509)



json size       95450 bytes

binary size   33868 bytes

~ 2.82 times compression

TILAugust 26, 2019by Dmitry Voronov

A simple way to distribute jobs in Sidekiq queues

This option implies that jobs of one context are executed sequentially in one queue, and jobs of different contexts in parallel in different queues.

Let's look at the following example.

There are investment funds for which we want to make time-consuming reporting calculations. Jobs for calculation within the same fund are carried out sequentially so that there are no errors in the calculations, jobs of different funds are performed in parallel.

We automate the distribution of jobs in queues so as not to specify a queue manually.

Specify the queues in the sidekiq.yml configuration file:


:queues:

  - fund_processor_0

  - fund_processor_1

  - fund_processor_2

  - fund_processor_3

  - fund_processor_4

It is important that the queues are numbered from 0.

Now, when starting the worker, we indicate in which queue we will set the job depending on the fund. To do this, we use the operation of obtaining the remainder from dividing the fund ID and the count of queues. So we get the queue number.


# 5 queues

# fund ID % count of queues = queue number

# 1 % 5 => 1

# 2 % 5 => 2

# 3 % 5 => 3

# 4 % 5 => 4

# 5 % 5 => 0

def queue_name(fund_id)

  queue_number = fund_id % 5

  "fund_processor_#{queue_number}"

end

Start the worker, indicating to him the received queue name.

This can be done using the Sidekiq API


Sidekiq::Client.push(

  'queue' => queue_name(fund_id),

  'class' => Fund::ReportCalculator,

  'args' => [fund_id]

)

LongreadsAugust 23, 2019by Igor Alexandrov

Metka Gem

Rails gem to manage tags with PostgreSQL array columns.
Read more
TILJuly 31, 2019by Igor Alexandrov

Migrate tags in Rails to PostgreSQL array from ActsAsTaggableOn

ActsAsTaggableOn is a swiss army knife solution if you need to add tags to your ActiveRecord model.

Just by adding one gem to your Gemfile and acts_as_taggable to the model you get everything you need: adding tags, searching for a model by tag, getting top tags, etc. However, sometimes you don't need all these.

In our project, we used acts_as_taggable to store tags for Note model. Then we displayed a list of notes on several pages with assigned tags and had autocompleted input for tags on Note form. Everything worked well, but since we use PostgreSQL, I decided to store tags as an array in Note model.

First of all, I added tags Array<String> column to Note, after this migrated actsastaggable tags to notes table with migration.


class MigrateNoteTags < ActiveRecord::Migration[5.2]

  def change

    execute <<-SQL

    UPDATE notes 

    SET tags = grouped_taggings.tags_array 

    FROM

      (

      SELECT

        taggings.taggable_id,

        ARRAY_AGG ( tags.NAME ) tags_array 

      FROM

        taggings

        LEFT JOIN tags ON taggings.tag_id = tags.ID 

      WHERE

        taggable_type = 'Note' 

      GROUP BY

        taggings.taggable_id 

      ) AS grouped_taggings 

    WHERE

      notes.ID = grouped_taggings.taggable_id

    SQL

  end

end

To have backward compatibility, I added Note#tag_list method:


def tag_list

  tags.join(', ')

end

The last thing is to add the ability to search for tags. Since there about 500k records in the Notes table, I decided to create an SQL view:


CREATE OR REPLACE VIEW note_tags AS



SELECT UNNEST

  ( tags ) AS name,

  COUNT ( * ) AS taggings_count 

FROM

  notes 

GROUP BY

  name 

That's it! It takes from 100ms to 150ms to search for tags in this view, which is fine for me.

If you have more significant data sets, then the best would be to create tags table and add triggers to notes table that will update tags on INSERT/UPDATE/DELETE.

LongreadsJuly 04, 2019by Igor Alexandrov

Deploy AnyCable with Capistrano and systemd

First of all, if you still don't know what is AnyCable than probably you never tried websockets in Rails. "WTF???" you probably will say, and you are right. Yes, we have ActionCable since Rails 5.x, but you really believe Ruby is suitable for real time web? I have bad news for you.
Read more
TILMarch 28, 2019by Dmitry Voronov

Create factory with an uploaded file in Rails rspec

To create a factory with an uploaded file in Rails rspec, needs to be included ActionDispatch::TestProcess module in the rails_helper.rb file to use the #fixture_file_upload method in the factories.


# spec/rails_helper.rb

include ActionDispatch::TestProcess

In a factory, when setting the attribute value, use the method #fixture_file_upload, specifying the path and file type.


# spec/factories/import_file.rb

FactoryBot.define do

  factory :import_file, class: ImportFile do

    data { fixture_file_upload 'spec/fixtures/test_file.pdf', 'application/pdf' }

  end

end

TILMarch 05, 2019by Alexander Spitsyn

How to conditionally render template in Grape resources?

Suppose you need to conditionally render some template in a given endpoint of a Grape resource. There is a env data in Grape::API class that represents Rack environment of the request. env is just a simple Ruby hash and we can pass the template name to it's api.tilt.template key to render that template:


env['api.tilt.template'] = 'foo/bar.json'

Then let's define a render method in Base resourсe:


class Base < Grape::API

  def self.inherited(subclass)

    super



    subclass.instance_eval do

      helpers do

        def render(template_name)

          env['api.tilt.template'] = template_name

        end

      end

    end

  end

end

Now we can use it in the following way:


class ContactsResource < Base

  desc 'Create contact', http_codes: [[201, 'Created']]



  post '/contacts' do

    contact = Contact.new(params[:contact])



    if contact.save

      render 'v1/contacts/show.json'

    else

      error!({ errors: contact.errors.full_messages }, 422)

    end

  end

end

TILMarch 05, 2019by Alexander Spitsyn

Append string to a route with a slash operator

If you need to append string to some route to build a path name you can use few ways:


Rails.root + 'foo/bar'

[Rails.root, 'foo/bar'].join('/')

"#{Rails.root}/foo/bar"

But also there is a Pathname#/ method, so you can append string to route using a slash:


Rails.root/'foo/bar' # => #<Pathname:/Users/alex/myproject/foo/bar>

Looks more readable!

TILFebruary 19, 2019by Alexander Budchanov

Safe Navigation vs Try in Rails (Part 2: Performance)

This Note is an extension of Safe Navigation vs Try in Rails (Part 1: Basic Differences)

Method try() comes from Active Support component of Ruby on Rails. Safe Navigation Operator is Ruby native feature.

Let's check performance!


require 'benchmark'



class Foo

  attr_accessor :name

end



foo = Foo.new

bar = nil



Benchmark.bm(35) do |x|

  x.report('Successful access: try(...): ')         { 1_000_000.times { foo.try(:name) } }

  x.report('Successful access: &.: ')               { 1_000_000.times { foo&.name } }

  x.report('Successful access: control sample: ')   { 1_000_000.times { foo.name } }

  x.report('Failed access: try(...): ')             { 1_000_000.times { bar.try(:nonexistent) } }

  x.report('Failed access: safe navigation: ')      { 1_000_000.times { bar&.nonexistent } }

end;nil


                                          user     system      total        real

Successful access: try(...):          0.498216   0.005748   0.503964 (  0.530010)

Successful access: &.:                0.062146   0.000943   0.063089 (  0.069714)

Successful access: control sample:    0.062411   0.001098   0.063509 (  0.069603)

Failed access: try(...):              0.172535   0.004374   0.176909 (  0.194386)

Failed access: safe navigation:       0.054141   0.001029   0.055170 (  0.065502)

Safe navigation is about 7 times faster than method try() for successful navigation and 3 times faster for unsuccessful.

TILFebruary 19, 2019by Alexander Budchanov

Safe Navigation vs Try in Rails (Part 1: Basic Differences)

There are some ways of preventing errors like undefined method for nil:NilClass.

  • Rails Method try(...)

  • Safe Navigation Operator (&.)

  • Logical operator && (AND)

Here is how these options look like:


user.try(:company).try(:name)


user&.company&.name


user && user.company && user.company.name

But there are some differences.

1. If model User hasn't relation compppany (it may be just a typo or renamed model relation/attribute):


user.try(:compppany).try(:name)

=> nil

You will receive nil and never been know about this typo.


user&.compppany&.name

=> NoMethodError: undefined method `compppany' for #<User:0x000000123456789>

and


user && user.compppany && user.compppany.name

=> NoMethodError: undefined method `compppany' for #<User:0x000000123456789>

Looks better!

2. If model User has relation company, but company is false. User.new(company: false):


user.try(:company).try(:name)

=> nil


user&.company&.name

=> NoMethodError: undefined method `name' for false:FalseClass

Safe Navigation recognized false. Awesome!


user && user.company && user.company.name

=> false

Hmmm, it does not look like we want.

3. Performance

Read the second part Safe Navigation vs Try in Rails (Part 2: Performance)

TILFebruary 12, 2019by Dmitry Voronov

How to delete polymorphic models cascade

If you use a polymorphic model in your Rails application, like in example


class Trade < ActiveRecord::Base

  has_many :gl_entries, as: :source, dependent: :destroy

end



class GlEntry < ActiveRecord::Base

  belongs_to :source, polymorphic: true

end

You will not be able to add the usual foreign keys for cascading delete records. But this can be implemented using a database.

To do this, you need to write a trigger in the database that will run the delete function for each record.


CREATE FUNCTION deleteGlEntriesOfTrade()

  RETURNS TRIGGER

  SET SCHEMA 'public'

  LANGUAGE plpgsql 

  AS $$

  BEGIN

    DELETE FROM gl_entries WHERE source_id = OLD.id AND source_type = 'Trade';

    RETURN OLD;   

  END;

  $$;



CREATE TRIGGER deleteTradesGlEntriesTrigger 

  BEFORE DELETE ON trades

  FOR EACH ROW EXECUTE PROCEDURE deleteGlEntriesOfTrade();

Create a migration and use :)

TILFebruary 11, 2019by Alexander Budchanov

Sidekiq Stats in Rails

If you use Sidekiq in your Rails project, you can watch tasks statistics through the web interface. Usually, you can find it here: https://yourdomain.com/sidekiq (but in your project path may be different).

If you don't have access there, you can get the same data in Rails console.

To get statistics (number of tasks) by each queue:


Sidekiq::Stats.new.queues

=> {"local_cache"=>0, "default"=>0, "ts_delta"=>0, "mailchimp"=>0, "recorder"=>0}

Global statistics are available this way:


Sidekiq::Stats.new.fetch_stats!

=> {:processed=>61390,

 :failed=>3220,

 :scheduled_size=>0,

 :retry_size=>0,

 :dead_size=>4,

 :processes_size=>1,

 :default_queue_latency=>0,

 :workers_size=>0,

 :enqueued=>0}

The same stats are available separately:


stats = Sidekiq::Stats.new

stats.processed # number of processed tasks

stats.failed # number of failed tasks

stats.enqueued # number of enqueued tasks

and etc.

Also Sidekiq::Stats allows to look historical data. You can specify period. Something like:


s = Sidekiq::Stats::History.new(2, Date.parse('2019-02-05'))

  • first argument: number of days,

  • second argument: date until which return stats (default value is today and can be omitted)


s.processed

=> {"2019-02-05"=>0, "2019-02-04"=>0}

s.failed

=> {"2019-02-05"=>0, "2019-02-04"=>0}

TILFebruary 04, 2019by Alexander Spitsyn

Getting rid of inefficient constantize

Trying to improve performance of one controller I've found the following:


def application_klass

  @application_klass ||= "application/#{params[:type]}".classify.constantize

end

There are Application::Rental and Application::Bridge models in our rails app, so the method above was used in two places:

  1. application_klass.to_s – to get the model name as a string

  2. application_klass.model_name.human – to get a class name without module prefix

I've refactored method in the following way:


def application_klass

  @application_klass ||= "application/#{params[:type]}".classify

end 

And it's occurrences now looks like this:

  1. application_klass – just call this method to get a model name

  2. application_klass.demodulize – using demodulize to split a string with ::

I've got rid of unnecessary constantize method, which is quite inefficient: just imagine that your application tries to find a constant in your project with the name specified in the string. See the benchmarks below to evaluate performance gains:


counter = 100_000

type = 'Rental'



Benchmark.bm(30) do |x|

  x.report('demodulize: ')                   { counter.times { "application/#{type}".classify.demodulize } }

  x.report('constantize.model_name.human: ') { counter.times { "application/#{type}".classify.constantize.model_name.human } }

end



                                     user     system      total        real

demodulize:                      3.560000   0.760000   4.320000 (  4.381891)

constantize.model_name.human:    8.500000   0.070000   8.570000 (  8.601007)

TILJanuary 31, 2019by Maksim Agafonov

Form_for and Rails attributes API

Attributes API allows you to have complex object with specified structure in fields. But how to make forms with this fields?

The answer is pretty simple: you work with them like with nested attributes.

Let’s say, we have class Provider with attribute address.


class Provider < ApplicationRecord

  attribute :address, Provider::Address::Type.new, default: Provider::Address.new

Address has following attributes:


class Provider::Address

  extend Dry::Initializer



  option :country, optional: true

  option :zip, optional: true

  option :city, optional: true

  option :address, optional: true

Form will look like this:


= form_for resource_provider, provider_url do |f|

  = f.fields_for :address do |ff|

      = ff.label 'Country'

      = ff.text_field :country, value: @provider.billing_address.country



      = ff.label 'Zip'

      = ff.text_field :zip, value: @provider.billing_address.zip



      = ff.label 'City'

      = ff.text_field :city, value: @provider.billing_address.city



      = ff.label 'Address'

      = ff.text_field :address, value: @provider.billing_address.address

In controller you have to permit parameters:


def provider_params

    params.require(:provider).permit(address: [:city, :zip, :country, :address])

end

And that’s all. Now, in action method you can assign attributes like this:


resource_provider.attributes = provider_params

resource_provider.save