How to set the password for Redis in Docker for Development?

I just spun a docker and found this error.

ERR Client sent AUTH, but no password is set

Then I realized I haven’t set the password. The traditional way is to create the redis.conf file and go with it but then again it is a lot of work as I would be maintaining another file.

Solution

If you are using Docker compose then simply override the CMD (command) like this:

command: redis-server --requirepass SUPER_SECRET

Worth Sharing?

Go Performance – A Checklist

Avoid Global Mutex

Don’t use a global mutex, keep it tied to structures. Go doesn’t have GIL and it is tempting to create a global mutex to simulate a similar behavior but then why use Go in the first place? You can tie a mutex to a structure to keep its impact as low as possible.

Avoid Defer Functions

Deffer functions are significantly expensive than the non-deferred ones. In the case of static functions, an improved version is compiled but it is still slower than direct functions.

Avoid recreate, reuse Variables

It is a usual way to create a variable in a loop. It takes an extra CPU cycle as well as it would keep Garbage Collection busy. Instead, use sync.Pool

Worth Sharing?

F`k it – Fix it – Psychology

I have just created a page Hire Me and told few friends and one asked me to come over and discuss a few projects his team is working where I can assist. Every single one of them was in Dot Net except one in Go. He is helping a renowned trader who has chosen to stay anonymous. A project is having code name OTAMIA.

Too Hot to Touch!

He stated that because of some inconsistency in an official Client of Kite Connect, the project is becoming annoying and unnecessary harder. His approach was not to touch the official API but use some duplicate structures to fix the issue or some conversions. He didn’t touch the official code but then again it wasn’t new. I created a framework on which every single Shopify App was build in SpiceGems and there, everyone worked on the framework decided not to touch what I wrote and this made a huge pain for everyone. I asked them why – they just said it is official (part of a framework)! My response was ‘F`k it Fix it’

F`k it – Fix it

The whole job of mine is to save time and pain. So we fixed the following issues or added feature (so far)

  1. Turned two packages into single – R.I.P compatibility.
  2. This enabled us to use a single structure for both packages.
  3. The error was coming as nil even when there was one – we fixed it.
  4. Data in converting JSON to STRUCT was giving blank (0000-00-000) dates – we added another known date format.
  5. We added a new flag to enable OI to be fetched.

All of the above changes were made within 2 hours but this removed all the unnecessary conversion, custom structure for proper JSON conversion and managing time column in our project. All of this so far had taken more than a week.

Be Brutal – Consultant’s Principal

Coding must be enjoyable and just because something is coming under the tag “Official” it doesn’t mean that there is no scope of improvement or you got to use that in the same way. If I can tell my team to f’k my framework, I can tell to f’k anyone’s. Knowledge and Tech both are evolving and we are all learning. Wherever you can save time and pain – just go ahead. Although, sometimes initial pain would save a lot of pain and time in the future. You will have to make a wise choice.

Open-Source KiteAPI

It was almost impossible for my client to let us make it opensource but he agreed and we have a code over here under MIT license.

https://github.com/VarunBatraIT/kiteapi

Worth Sharing?

Cannot assign to struct field in map

Understanding Maps in Go

Have you ever come across an error “Cannot assign to struct field in map”? So what is the issue here?

Imagine that we have a structure:

package main

type Product struct {
	Name  string
	Image string
	Cdn   string
	Price float64
}

func main() {

	products := make(map[int]Product)

	products[0] = Product{"Blue Diamond Book", "https://localhost/split/bdb.png", "https://localhostcdn/split/bdb.png", 120.00}
	// Problem
	products[0].Price = 121.00

}

The Error

If we have to reassign value to a variable like this:

products[0].Price = 121.00

We’ll get an error: “Cannot assign to struct field in map”

Pointer Map Problem in Golang

The reason for this panic is, the map is a pointer to value and Go checks the key, is it there or not. If you had used a map, you must have seen:

thisProduct, ok := products[0]

thisProduct” will get the Product structure’s object and “ok” will get a bool value, that this key exists or not

So we have to assign the pointer to a variable.

// Solution
	if thisProduct, ok := products[0]; ok {
		thisProduct.Price = 121.00
		products[0] = thisProduct
	}

Now, we can access and reassign the value of the Student structure’s object:
thisProduct.Price = 121.00
After making the required adjustments, this thisProduct can be assigned back to the same pointer.

Worth Sharing?

How to use SSL with OCSP with GIN Framework in Go

The GIN framework comes up with the SSL middleware autotls which requests the SSL certificate on request basis. However, it lacks the cool feature of OCSP

However, the CertMagic which generates SSL certificate during initializations – so no surprises here and does support OCSP.


app := gin.Default()
...

// Assuming SSL_DOMAIN has csv domains like varunbatra.com,www.varunbatra.com
domains := os.Getenv("SSL_DOMAIN")
domainsMap := strings.Split(domains, ",")
err := certmagic.HTTPS(domainsMap, app)
if err != nil {
	log.Fatal(err)
}

Worth Sharing?

Easy Ngrok with Docker

If you are using localhost as a development platform, you are probably aware of ngrok. You might have installed it in your machine, along with some dependencies. While I have focused on stateless development machine, it is important for me and developers like me to have everything portable. Now I have ngrok portable with a simple alias:

function expose-ngrok() {
  docker run --rm --net=host -e NGROK_PORT="$1" wernight/ngrok
}

Then navigate to localhost:4040 to find the url assigned to you.

Worth Sharing?

How to Get Public IP Address in Linux

Problem with WGET/CURL

You might have a seen a lot of wget/curl related way to get an IP address. One of my clients was using it and it looks like somehow he was blocked. Now it might have not been the case if he was spoofing but assume that he would get blocked eventually, I used dig with opendns (over Google).

Blocking isn’t one way – may be the service is down – it happened to us all isn’t it?

With dig, system is running for a year and my Client hired me for another project and asked me how I did it. So I thought I should share it to readers as well.

Most Reliable way is through dig

dig +short myip.opendns.com @resolver1.opendns.com
dig TXT +short o-o.myaddr.l.google.com @ns1.google.com | awk -F'"' '{ print $2}'

A little less reliable way is through host

host myip.opendns.com resolver1.opendns.com | grep "myip.opendns.com has" | awk '{print $4}'

Worth Sharing?

How to make MySQL inside Docker Production Ready?

It is easy to simply spin a new MySQL and assume that it is ready for the production. This can’t be further from the preparation of disaster.

Few days back I wrote about having a infrastructure docker file to do your development and I got a comment was it ready for production. Today that ‘No’ is about to change to ‘Yes’.

When application grows, the table grows and we need more in-memory, more innodb instances and more threads. However, default MySQL configuration is not changed for last 8-10 years. Machine has got faster, MySQL default config didn’t catch up to that. Thus it is important to override configurations.

This is one example:

docker-compose.yml

version: '3.2'
networks:
  dual-localhost:
    driver: bridge
services:
  mysql:
    image: mysql:5.8
    restart: always
    volumes:
        - type: bind
          source: ./mysql
          target: /var/lib/mysql
        - type: bind
          source: /var/log/mysql/
          target: /var/log/mysql/
        - type: bind
          source: ./mysql.cnf
          target: /etc/mysql/mysql.conf.d/mysql.cnf
    environment:
        - MYSQL_ROOT_PASSWORD=some_weird_password
    networks:
      - dual-localhost
[mysqld_safe]
socket		= /var/run/mysqld/mysqld.sock
nice		= 0

[mysqld]
#
# * Basic Settings
#
user		= mysql
pid-file	= /var/run/mysqld/mysqld.pid
socket		= /var/run/mysqld/mysqld.sock
port		= 3306
basedir		= /usr
datadir		= /var/lib/mysql
tmpdir		= /tmp
lc-messages-dir	= /usr/share/mysql
skip-external-locking
#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
#bind-address		= 0.0.0.0
#skip-networking #insecure
#bind-address = 127.0.0.1
#
# * Fine Tuning
#
key_buffer_size		= 16M
max_allowed_packet	= 512M
thread_stack		= 192K
thread_cache_size       = 64
innodb_read_io_threads = 2
innodb_write_io_threads = 2
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
myisam-recover-options  = BACKUP
#max_connections        = 100
#table_cache            = 64
#thread_concurrency     = 10
#
# * Query Cache Configuration
#
query_cache_limit	= 8M
tmp_table_size      = 32M
query_cache_size        = 32M
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
general_log_file        = /var/log/mysql/mysql.log
#general_log             = 1
#
# Error log - should be very few entries.
#
log_error = /var/log/mysql/error.log
#
# Here you can see queries with especially long duration
# slow-query-log=1
# slow-query-log-file=/var/log/mysql/mysql-slow.log

#log-queries-not-using-indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
#       other settings you may need to change.
server-id		= 10
log_bin			= /var/log/mysql/mysql-bin.log
expire_logs_days	= 10
binlog_row_image=minimal
max_binlog_size   = 256M
binlog_cache_size = 2M
binlog_rows_query_log_events = on

relay-log               = /var/log/mysql/mysql-relay-bin.log

innodb_log_file_size = 512M
#binlog_do_db		= include_database_name
#binlog_ignore_db	= include_database_name
#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
#
# * Security Features
#
# Read the manual, too, if you want chroot!
# chroot = /var/lib/mysql/
#
# For generating SSL certificates I recommend the OpenSSL GUI "tinyca".
#
# ssl-ca=/etc/mysql/cacert.pem
# ssl-cert=/etc/mysql/server-cert.pem
# ssl-key=/etc/mysql/server-key.pem
collation_server=utf8mb4_unicode_ci
character_set_server=utf8mb4

wait_timeout = 500
interactive_timeout = 28800

open_files_limit = 1024000
skip-name-resolve

join_buffer_size = 512K

innodb_buffer_pool_size = 3174M
innodb_buffer_pool_instances = 2
innodb_stats_persistent_sample_pages = 100
innodb_stats_transient_sample_pages = 24
innodb_rollback_on_timeout = on

Worth Sharing?

Completing WordPress Health Check inside Docker

Missing Modules

It is very tempting to simply use the official WordPress Docker image. However, new feature HEALH CHECK in 5.X – you will observe a recommendation – something like following:

Missing Modules in WordPress Docker

To sum up – bcmath, exif and imagick are not installed in wordpress docker image by default.

Resolution

The same way we added Imagick in PHP Docker, You can create your own Dockerfile instead of using image directly as following:

FROM wordpress:5.2.0-php7.3-apache
# Better WordPress Health-Check
# Install Imagick
RUN apt-get update && apt-get install -y libmagickwand-dev --no-install-recommends && rm -rf /var/lib/apt/lists/*
RUN printf "\n" | pecl install imagick
RUN docker-php-ext-enable imagick
# Install bcmath and exif
RUN docker-php-ext-install bcmath exif

Great Job! Everything is running smoothly here.

Worth Sharing?