TIL from the Rocketship

TILMay 28, 2020by Vladimir Nikonorov

Dynamic form validation with dry-validation gem

There can be situations when we need dynamic validation of incoming params.

For example, we have a form with many fields that are stored in our database. Form fields can be added, deleted and customized by users. We would like to validate this form using convenient dry-validation gem syntax. Unfortunately, in dry-validation there is no way to do this:


class FormContract < Dry::Validation::Contract

  option :form_fields



  params do

    form_fields.each do |form_field|   # form_fields variable is not available in params block

      required(form_field.name).filled(:string)

    end

  end

end

We can validate our form in the following way:


class FormContractBuilder

  def self.schema(form_fields)

    form_contract = Class.new(Dry::Validation::Contract) {

      params do

        form_fields.each do |form_field|

          required(form_field.name).filled(:string)

        end

      end

    }



    form_contract.new

  end

end



FormContractBuilder.schema(form.fields).call(params)

TILMarch 22, 2020by Dmitry Sokolov

Absolute imports in Next.js

When I was working on my first project with Next.js I spent a lot of time to find how to use absolute imports. The solution turned out to be quite simple. You need to indicate this resolve option in your next.config.js file:


module.exports = {

  webpack(config) {

    config.resolve.modules.push(__dirname)

    return config;

  },

}

Or if you want to add absolute path only for one directory you can use alias:


module.exports = {

  webpack(config) {

    config.resolve.alias['components'] = path.join(__dirname, 'components');

    return config;

  },

}

TILFebruary 19, 2020by Igor Alexandrov

Migrating user passwords from Django to Ruby

One of our clients asked us to migrate his existing Django application to Ruby. A part of this process was a migration of an existing users database.

Of course we had to find a way to use existing crypto passwords from Django and not asking our users to reset them

Passwords in Django are stored as a string that consists of four parts splitted with a dollar sign.


<algorithm>$<iterations>$<salt>$<hash>

By default, Django uses the PBKDF2 algorithm with a SHA256 hash. I found a rather outdated Ruby gem that implements a Password-Based Key-Derivation Function and gave it a try.


def migrate_django_password_seamlessly(user, password)

  alg, iteration, salt, django_password = user.crypted_django_password.split('$')

  attempt = Base64.encode64(

    PBKDF2.new(password: password, salt: salt, iterations: 36000).bin_string

  ).strip



  # check if hash of user provided password equals to the password in a database

  if attempt == django_password

    user.update(password: password, password_confirmation: password)

  end

end

The method above is a part of user sign in service and called only if crypted_django_passwordcolumn from users table is not null.

TILJanuary 06, 2020by Dmitry Sokolov

Problem with download file in Google Chrome

I encountered such a problem when I tried to download base64 decoded file with a size over 1mb in Chrome.

I had a function like this:


export const downloadBase64File = (fileData, fileName) => {

  const fileUrl = `data:application/octet-stream;base64,${fileData}`;

  const link = document.createElement('a');

  link.href = fileUrl;

  link.setAttribute('download', fileName);

  document.body.appendChild(link);

  link.click();

  link.remove();

};

It worked good in FireFox but didn't work in Google Chrome. And I found the following solution:


const urlToFile = (url, filename) => {

  return fetch(url)

    .then((res) => {

      return res.arrayBuffer();

    })

    .then((buf) => {

      return new File([buf], filename);

    });

};



export const downloadBase64File = (fileData, fileName) => {

  const fileUrl = `data:application/octet-stream;base64,${fileData}`;



  urlToFile(fileUrl, fileName).then((file) => {

    const blob = new Blob([file], { type: 'application/octet-stream' });

    const blobURL = window.URL.createObjectURL(blob);

    const link = document.createElement('a');

    link.href = blobURL;

    link.setAttribute('download', fileName);

    document.body.appendChild(link);

    link.click();

    link.remove();

  });

}

It works correctly for all browsers.

TILDecember 26, 2019by Dmitry Sokolov

Environment variables in NEXT js

How to get env variables to your app code in NEXT.js application?

It’s not a big deal, but if you set ‘NODE_ENV’ for env in next.config.js it will not work correctly.

Anytime you will have ‘production’ (default value at build time) for any environments (staging, integration, production).

Just use another name instead ‘NODE_ENV’, for example ‘ENV’.


// next.config.js



require('dotenv').config()

module.exports = {

  env: {

    ENV: process.env.NODE_ENV,

  },

}

It will be available in your app code


export default () => <div>{process.env.ENV}</div>

TILNovember 26, 2019by Igor Alexandrov

How to add HTTP Basic auth to Amber application

As you already may know, one of our projects have a Crystal application in production. It is created with Amber framework and works just perfect.

The only thing that I don't personally like in Amber is not clear and sometimes outdated documentation, after about 11 years with Rails I still think that Rails Guides are the number one developer documentation in the world.

I had a task to add HTTP Basic auth to a couple of URLs in Amber application, and after studying documentation found that Amber doesn't provide necessary Pipe out of the box. Ok, next place to search for the answer was Gitter and after about an hour Dru Jensen helped me with a code example.

Amber uses internally HTTP::Handler for Pipes as well as Kemal does for Middlewares, so we can easily use code from Basic Auth for Kemal.


# src/pipes/http_basic_auth_pipe.cr



require "crypto/subtle"



class HTTPBasicAuthPipe

  include HTTP::Handler

  BASIC = "Basic"

  AUTH = "Authorization"

  AUTH_MESSAGE = "Could not verify your access level for that URL.\nYou have to login with proper credentials"

  HEADER_LOGIN_REQUIRED = "Basic realm=\"Login Required\""



  property credentials : Credentials?



  def initialize(@credentials : Credentials)

  end



  def initialize(username : String, password : String)

    initialize({ username => password })

  end



  def initialize(hash : Hash(String, String))

    initialize(Credentials.new(hash))

  end



  def initialize

    if ENV["HTTP_BASIC_USERNAME"]? && ENV["HTTP_BASIC_PASSWORD"]?

      initialize(ENV["HTTP_BASIC_USERNAME"], ENV["HTTP_BASIC_PASSWORD"])

    end

  end



  def call(context)

    if credentials

      if context.request.headers[AUTH]?

        if value = context.request.headers[AUTH]

          if value.size > 0 && value.starts_with?(BASIC)

            return call_next(context) if authorized?(value)

          end

        end

      end

      headers = HTTP::Headers.new

      context.response.status_code = 401

      context.response.headers["WWW-Authenticate"] = HEADER_LOGIN_REQUIRED

      context.response.print AUTH_MESSAGE

    else

      call_next(context)

    end

  end



  private def authorized?(value)

    username, password = Base64.decode_string(value[BASIC.size + 1..-1]).split(":")

    credentials.not_nil!.authorize?(username, password)

  end



  class Credentials

    def initialize(@entries : Hash(String, String) = Hash(String, String).new)

    end



    def authorize?(username : String, given_password : String) : String?

      test_password = find_password(username, given_password)

      if Crypto::Subtle.constant_time_compare(test_password, given_password)

        username

      else

        nil

      end

    end



    private def find_password(username, given_password)

      # return a password that cannot possibly be correct if the username is wrong

      pw = "not #{given_password}"



      # iterate through each possibility to not leak info about valid usernames

      @entries.each do |(user, password)|

        if Crypto::Subtle.constant_time_compare(user, username)

          pw = password

        end

      end



      pw

    end

  end

end

And in routes.cr:


  # ...

  pipeline :api do

    # ...

    plug HTTPBasicAuthPipe.new

  end

  # ...

TILOctober 29, 2019by Alexander Blinov

onChange trigger example with React

Recently, I faced the issue. There is a form with some fields. Each field has several functions; functions do something. For example, one function writes the field name and its value into the object. Then the customer asks to add the buttons with specified values. When the button is clicked, the relevant field should update its value.

Here is the trigger hack that calls the onChange function of that field.


function triggerInput(enteredName, enteredValue) {

  const input = document.getElementById(enteredName);



  const lastValue = input.value;

  input.value = enteredValue;

  const event = new Event("input", { bubbles: true });

  const tracker = input._valueTracker;

  if (tracker) {

    tracker.setValue(lastValue);

  }

  input.dispatchEvent(event);

}

Live example link

TILOctober 18, 2019by Alexey Belousov

Activity Indicator in SwiftUI

There are a lot of elegant solutions for typical tasks in web development. One of such solutions is a Loader.css — library of animated activity indicators. It is actively ported to different languages and platforms. The following libraries are available for iOS:

  • DGActivityIndicatorView (Objective-C);

  • NVActivityIndicatorView (Swift).

I wanted to implement one of the styles (number 29 in the NVActivityIndicatorView list) on SwiftUI:


struct ActivityIndicator: View {

  @State private var isAnimating: Bool = false



  var body: some View {

    GeometryReader { (geometry: GeometryProxy) in

      ForEach(0..<5) { index in

        Group {

          Circle()

            .frame(width: geometry.size.width / 5, height: geometry.size.height / 5)

            .scaleEffect(!self.isAnimating ? 1 - CGFloat(index) / 5 : 0.2 + CGFloat(index) / 5)

            .offset(y: geometry.size.width / 10 - geometry.size.height / 2)

          }.frame(width: geometry.size.width, height: geometry.size.height)

            .rotationEffect(!self.isAnimating ? .degrees(0) : .degrees(360))

            .animation(Animation

              .timingCurve(0.5, 0.15 + Double(index) / 5, 0.25, 1, duration: 1.5)

              .repeatForever(autoreverses: false))

        }

      }.aspectRatio(1, contentMode: .fit)

        .onAppear {

          self.isAnimating = true

        }

  }  

}

You can use this component as follows:


let kPreviewBackground = Color(red: 237/255.0, green: 85/255.0, blue: 101/255.0)



struct ContentView: View {

  var body: some View {

    ZStack {

      kPreviewBackground

        .edgesIgnoringSafeArea(.all)



      VStack {

        ActivityIndicator()

          .frame(width: 50, height: 50)

        }.foregroundColor(Color.white)

      }

  }

}

This example can be used as a starting point for creating a universal activity indicator with support for different display styles.

Source Code

Source Code available on GitHub Gists.

See Also

TILOctober 10, 2019by Maxim Romanov

How quickly and easily run a local server with fake api data (mocks)?

Sometimes you need to develop a frontend part of a project without a ready-made api, knowing only its structure. In this case, using json-schema-faker, you can generate fake data (mocks) and deploy it to your local server.

Firstly u will need to install json-schema-faker package


yarn add json-schema-faker

Then open the package.json file and add scripts with the following


// ...

"scripts": {

  // ...

  "start-mockapi": "json-server --watch ./mocks/api/db.json --port 3001",

  "generate-mock-data": "node ./generateMockData",

}

After installation, you will need to describe the structure in ./mocks/dataSchema.js of future mocks. You can find more information here.


const schema = {

  reports: {

    type: 'array',

    minItems: 5,

    maxItems: 10,

    items: {

      id: {

        type: 'integer',

        unique: true,

        minimum: 1,

        maximum: 1000,

      },

      title: {

        enum: ['production', 'azure data', 'azure data 2'],

      },

      logo: 'https://picsum.photos/200'

    },

  },

}



module.exports = schema;

Copy paste script for generating mock data from here in ./generateMockData.js and run the following


yarn generate-mock-data && yarn start-mockapi

TILOctober 10, 2019by Igor Alexandrov

Double splat arguments in Crystal

In Crystal, as well as in Ruby you can use double splat arguments. Unfortunately they behave a bit different.


def foo(**options)

  baz(**options, a: 1)

end



def baz(**options)

  puts options

end



foo(b: 2, a: 3) # {:b=>2, :a=>1} 

This code in Ruby works as it should. If we try the same in Crystal (https://play.crystal-lang.org/#/r/7r0l), we got an error:


error in line 2

Error: duplicate key: a

This happens because **options is a NamedTuple and it cannot have duplicate keys. I found that using NamedTuple#merge can be a workaround (https://play.crystal-lang.org/#/r/7s1c):


def foo(**options)

  baz(**options.merge(a: 1))

end



def baz(**options)

  puts options

end



foo(b: 2, a: 3) # {:b=>2, :a=>1} 

Hack!

TILOctober 03, 2019by Eugene Komissarov

Save your links from phishers.

While Pull Request fixing issue with OWASP Tabnabbing is still open... since Feb 16, 2017, our shiny Rails applications are in danger. But wait no longer! Just put code like this:


# frozen_string_literal: true



module ActionView

  module Helpers #:nodoc:

      module UrlHelper

      # Same as #link_to, but also adds rel="nofollow" and rel="noopener" if target="_blank"

      #   rel='noopener' is added to mitigate OWASP Reverse Tabnabbing

      #

      #   external_link_to "External link", "http://www.rubyonrails.org/", target: "_blank"

      #   # => <a href="http://www.rubyonrails.org/" target="_blank" rel="nofollow noopener">External link</a>

      def external_link_to(name = nil, options = nil, html_options = nil, &block)

        html_options, options, name = options, name, yield if block_given?

        html_options ||= {}

        html_options.stringify_keys!



        html_options['rel'.freeze] = "#{html_options['rel'.freeze]} nofollow".lstrip

        html_options['rel'.freeze] = "#{html_options['rel'.freeze]} noopener".lstrip if html_options['target'.freeze] == '_blank'.freeze

        link_to(name, options, html_options)

      end

    end

  end

end

into one of initializers and enjoy bit of safety. This will add new url helper external_link_to to your disposal, that will mitigate Reverse Tabnabbing endangering your application.

Or, if you feel adventurous today... lets patch link_to itself!


# frozen_string_literal: true



module ActionView

  module Helpers #:nodoc:

    module UrlHelper



      def link_to(name = nil, options = nil, html_options = nil, &block)

        html_options, options, name = options, name, block if block_given?

        options ||= {}





        html_options = convert_options_to_data_attributes(options, html_options)



        html_options['rel'.freeze] = "#{html_options['rel'.freeze]} nofollow".lstrip

        html_options['rel'.freeze] = "#{html_options['rel'.freeze]} noopener".lstrip if html_options['target'.freeze] == '_blank'.freeze



        url = url_for(options)

        html_options["href".freeze] ||= url



        content_tag("a".freeze, name || url, html_options, &block)

      end

    end

  end

end

Even better! Now all your existing links with target="_blank" gonna be safe from nasty fishers.

Happy coding and stay safe.

TILSeptember 18, 2019by Alexander Spitsyn

Rewind Files Before Reading

Suppose you have a form with a file input and you need to forward that file to an external service through your Rails app. Params will look like this:


{ "file"=> #<ActionDispatch::Http::UploadedFile:0x00007fce9a8af140 @tempfile=#<Tempfile:/path/to/the/file.xlsx>, ... > }

The external service can not be reached for the first time sometimes, so you decided to set some retry count.


retry_count = 5

while retry_count > 0

    begin

        HTTPClient.new.post(url, params, headers)

    rescue ExternalServiceUnavailable => e

        retry_count -= 1

    end

end

The code above uses HTTPClient gem for making http requests and it will raise the following error in case there will be at least one retry:

ArgumentError: Illegal size value: #size returns 154139 but cannot read

The thing is that HTTPClient uses IO#read to read the file by chunks for making POST request. The lineno pointer reaches the end of the file after read, and you can not read bytes from the I/O stream anymore:


params['file'].tempfile.eof? # => true

To fix that, use IO#rewind – it positions IO to the beginning of input, resetting lineno to zero. Just add checking the file for eof before each request and rewind it if needed:


retry_count = 5

while retry_count > 0

    begin

        params['file'].tempfile.rewind if params['file'].tempfile.eof? # check eof

        HTTPClient.new.post(url, params, headers)

    rescue ExternalServiceUnavailable => e

        retry_count -= 1

    end

end

TILSeptember 17, 2019by Alexander Budchanov

Renaming keys in PostgreSQL JSON / Migration template for gem Recorder

In our project, we used gem Recorder. It saves changes in jsonb format.

Once we need to rename a column in model Contact, but we didn't want to lose logs.

I introduce you template of migration for this case.


class RenameOldColumnToNewColumn < ActiveRecord::Migration[5.2]

  def up

    rename_column :contacts, :old_column, :new_column



    execute <<~SQL

      UPDATE recorder_revisions SET data = jsonb_set(data #- '{changes,old_column}', '{changes,new_column}', data#>'{changes,old_column}') WHERE data#>'{changes}'?'old_column';

    SQL

  end



  def down

    rename_column :contacts, :new_column, :old_column



    execute <<~SQL

      UPDATE recorder_revisions SET data = jsonb_set(data #- '{changes,new_column}', '{changes,old_column}', data#>'{changes,new_column}') WHERE data#>'{changes}'?'new_column';

    SQL

  end

end

You can simple replace contacts, new_column and old_column to your table name and columns names.

TILSeptember 18, 2019by Andrey Morozov

Strip Whitespace from Heredocs in Ruby/Rails

UPDATE!

Simple way - it's use:


class Template

  def self.base

    <<~TEXT

      Lorem Ipsum is simply dummy

      Lorem Ipsum is simply dummy

    TEXT

  end

end



> Template.base

=> "Lorem Ipsum is simply dummy\nLorem Ipsum is simply dummy\n"

I forgot about <<~, and that's why I got this TIL bike.


You use heredoc in Ruby/Rails app?

Then you're familiar with the problem..


class Template

  def self.base

    <<-TEXT

      Lorem Ipsum is simply dummy

      Lorem Ipsum is simply dummy

      Lorem Ipsum is simply dummy

    TEXT

  end

end



> Template.base

=> "      Lorem Ipsum is simply dummy\n      Lorem Ipsum is simply dummy\n      Lorem Ipsum is simply dummy\n"

hmm...

I don't want to have so many gaps in the text.

Maybe use them:


class Template

  def self.base

    <<-TEXT

Lorem Ipsum is simply dummy

Lorem Ipsum is simply dummy

    TEXT

  end

end



> Template.base

=> "Lorem Ipsum is simply dummy\nLorem Ipsum is simply dummy\n"

It's good, but the code looks like 💩.

This problem can be solved by using activesupport #strip_heredoc:


class Template

  def self.base

    <<-TEXT.strip_heredoc

      Lorem Ipsum is simply dummy

      Lorem Ipsum is simply dummy

    TEXT

  end

end



> Template.base

=> "Lorem Ipsum is simply dummy\nLorem Ipsum is simply dummy\n"

If you don't have active_support, use refine String with #strip_heredoc which will give similar result:


module Utils 

  refine String do 

    def strip_heredoc

      gsub(/^#{scan(/^[ \t]*(?=\S)/).min}/, "".freeze)

    end

  end

end



class Template

  using Utils

  def self.base

    <<-TEXT.strip_heredoc

      Lorem Ipsum is simply dummy

      Lorem Ipsum is simply dummy

    TEXT

  end

end



> Template.base 

=> "Lorem Ipsum is simply dummy\nLorem Ipsum is simply dummy\n"

TILSeptember 17, 2019by Alexander Budchanov

Rails 5.2 changes in callbacks

In version 5.1 you may see deprecation warnings in after_save callbacks (related to changes in ActiveRecord::Dirty module).

But since 5.2 these changes were applied.

For examples, I will use Rails 4.2.11 and Rails 5.2.3 and model User with email attribute. Let's do:


u = User.new(email: 'old@domain.com')

u.save

u.email = 'new@domain.com'

u.save

and look at after_save callback in time of last save.

1. attribute_changed?

Rails 4


> email_changed?

=> true

Rails 5.2


> email_changed?

=> false

but you can use saved_changes?


> saved_change_to_email?

=> true

2. changed?

Rails 4


> changed?

=> true

Rails 5.2


> changed?

=> false

but you can use saved_changes?


> saved_changes?

=> true

3. changes

Rails 4


> changes

=> {"email"=>["old@domain.com", "new@domain.com"]}

Rails 5.2


> changes

=> {}

but you can use saved_changes


> saved_changes

=> {"email"=>["old@domain.com", "new@domain.com"]}

4. previous_changes

Rails 4


> previous_changes

=> {"email"=>[nil, "old@domain.com"]}

Rails 5.2

Now, this method returns the changes that were just saved (like saved_changes).


> previous_changes

=> {"email"=>["old@domain.com", "new@domain.com"]}

this method has no replacement.

TILSeptember 16, 2019by Andrey Morozov

Simple way to get all values from hash

In the ruby from the box, we can't find all the hash values if it's nested.

I suggest an easy way to find all the values using recursion.

Example hash:


hash = {

  a: 2,

  b: { c: 3, d: 4, e: {f: 5}}

}


> hash.values

=> [2, {:c=>3, :d=>4, :e=>{:f=>5}}]

That's not an option, we need all the values.


def deep_values(array = [], object:)

  object.each do |_key, value|

    if value.is_a?(Hash)

      deep_values(array, object: value)

    else

      array << value

    end

  end

  array

end



> deep_values(object: hash)

=> [2, 3, 4, 5]

If you run the benchmark with this data, we get the following data:


>  puts Benchmark.measure { 100_000.times { hash.values } }

=> 0.028920   0.002643   0.031563 (  0.032759)



>  puts Benchmark.measure { 100_000.times { deep_values(object: hash ) } }

=> 0.140439   0.003318   0.143757 (  0.146637)

TILSeptember 13, 2019by Igor Alexandrov

How to parse CSV with double quote (") character in Crystal

We use microservice written in Crystal to parse large CSV files (about 1.5Gb). Some rows in these files may contain no closed " characters:


,Y,FEDERAL NATIONAL MORTGAGE ASSOCIATION "F,,

With Crystal default CSV parse settings this row and everything after it won't be parsed correctly because DEFAULT_QUOTE_CHAR constant is equal to ". Of couse you can override quote_char param in CSV contstructor with something that cannot be found in your document.

From my point of view the best is to use zero byte which is '\u0000' in Crystal.


csv = CSV.new(file, headers: true, strip: true, quote_char: '\u0000')

while csv.next

  # ... 

end

Hack!

TILSeptember 13, 2019by Dmitry Voronov

How to use Rails translations for Reform attributes

If you use Reform for the form objects and want the translations in your Rails application to work as with ActiveRecord objects, then you can add ActiveModel::Translation module to the form class or base class and specify the method with the key to translations.


class BaseForm < Reform::Form

  extend ActiveModel::Translation



  def self.i18n_scope

    :forms

  end

end


class UserForm < BaseForm

  feature Coercion



  property :login

  property :change_password, virtual: true, type: Types::Form::Bool

  property :password

  property :password_confirmation



  validation do

    required(:login).filled

  end



  validation if: ->(_) { change_password } do

    required(:password).confirmation

  end

end

Set translations for form fields.

You can move them to a new file config/locales/forms.yml.


en:

  forms:

    attributes:

      user_form:

        login: Enter login

        change_password: Change password?

        password: Enter password

        password_confirmation: Confirm password

Now, translations will be used by the simple form or you can call them manually by UserForm.human_attribute_name.

TILSeptember 05, 2019by Alexander Ivlev

Redux async actions. Tracking loading and errors with React hooks.

If you use redux and async actions, then you probably had to deal with a situation where you need to monitor the loading and error status of this action. With the advent of hooks, it became possible to conveniently transfer this logic to one block and use it everywhere.


import { useState, useCallback } from 'react';

import { useDispatch } from 'react-redux';



function useAsyncAction(action, dependeces = []) {

  const dispatch = useDispatch();

  const [loading, setLoading] = useState(false);

  const [isError, setIsError] = useState(false);



  const asyncAction = useCallback(

    (...args) => {

      async function callback() {

        setLoading(true);

        try {

          const res = await dispatch(action(...args));

          setIsError(false);

          setLoading(false);

          return res;

        } catch (e) {

          setLoading(false);

          setIsError(true);

          return e;

        }

      }

      callback();

    },

    [action, ...dependeces],

  );



  return [asyncAction, loading, isError];

}

Now you can use this hook in your functional component.


// …

  const [load, loading, isError] = useAsyncAction(() => loadActivityRequest(applicationId), [applicationId]);

// …

  load();

TILSeptember 04, 2019by Vitaly Platonov

Postgres. Search Array type columns

Say we have a table with a column of an array type. At some point, we want to be able to select records with a specific value(s) which the array column may have.

Here are three ways to do different kinds of searches.

1) Use ANY operator when searching with one value:


SELECT * FROM mytable WHERE 'first_type' = ANY(types_column);

2) Go with the "contains" operator (“@>”) when you look for a specific set of values (the order of values doesn’t matter):


SELECT * FROM mytable WHERE types_column @> '{"first_type", "second_type"}';

The values “first_type” and “second_type" must be in the types_column column for a record to be selected.

3) Whenever you need to search any values that a column may have - use the "overlap" operator (“&&”)


SELECT * FROM mytable WHERE types_column && '{"first_type", "second_type"}';

One of the values “first_type” or “second_type" must be in the types_column column for a record to be selected.