Your browser doesn't support the features required by impress.js, so you are presented with a simplified version of this presentation.

For the best experience please use the latest Chrome, Safari or Firefox browser.

Automation strategies 
for deploying Grails 
from Dev to Prod

Eric Helgeson

@nulleric

Who am I?

Grails Consulting at Agile Orbit

http://www.agileorbit.com

Practical Grails 3


https://www.grails3book.com

Sproutary.com

Online software for Pre-Schools and Day Cares.
https://sproutary.com

About this Talk

Agenda

  • Everything as Code
  • Building for Production
  • Infrastructure testing
  • Operating in production

Information share

  • Give you some new ideas to try
    • Incremental & Continuous improvements
  • Hear ideas & experiences from you

Poll - Show of hands

Continuously Build (Push code, it builds)

Continuously Deploy (master == prod)

Infrastructure improvements are something:

  • Everyone on the team helps with
  • Other people’s jobs

Everything as code

Building

Codify your build

  • Lots of options!

SaaS

  • Easy to setup & use
  • Doesn’t fit everyone’s needs
  • Travis, CircleCi

Demo!

Travis

Jenkins (of course)

Jenkins

Building a workflow

ChatDev?

  • Keep development flow going, push notifications when action required.
  • Notify dev in Slack when:
    • Build fails
    • Build passes
      • Ping you in slack with deployment options
    • Someone assigns you a ticket/bug/review

Code reviews as code

Things to stop manually reviewing

  • Indentation
  • Spaces vs. Tabs
  • Unimportant things!

Things to review

  • Functionality

Linters

  • CodeNarc
    • eg: java.util.Date == bad? - make a rule!

Build steps

Abstract specifics

Good ./gradlew test integrationTest jaco assemble
Better ./gradlew assemble
Best ./build.sh - Builds change - allow devs to change build steps on their branch.

Artifacts

Take availability to your own hands

  • Can you deploy if maven central/grails repo is down?

Demo!

Lets setup an artifactory server.

Versioning

  • Version based off git branch & short-sha

Demo!

info/info

Dev & Test as code

Allow devs to deploy a prod like env

  • locally (Vagrant/Docker/etc)
  • On demand SaaS (EB/AWS/etc)
  • Document what is different
    • 1 hadoop node vs 3, etc
  • Living datasets, easy access

Demo!

Vagrant

Building for Production

Jars are our containers

Create single executable artifact

  • ./gradlew assemble
  • java -jar myapp.jar

Why?

  • Isolate JVM (class loader, etc)
  • Single artifact
  • Bundle different containers for different apps
  • Can be executable!
  • Deployment
    • Easy (s)cp && restart
    • Blue/Green
    • Hot deploys never work

Demo!

executable

Grails notes

Configuration

External Config

  • External config plugin
    • Render .groovy file from Config Management
  • Use K/V store such as Consul, etc, spring-cloud, zk
  • Inject ENV vars
    environments {
    production {
      dataSource {
          dbCreate = "none"
          driverClassName = "org.postgresql.Driver"
          dialect = "org.hibernate.dialect.PostgreSQL9Dialect"
          url = "jdbc:postgresql://${System.getenv('DB_HOST')}:${System.getenv('DB_PORT')}/${System.getenv('DB_NAME')}"
          username = "${System.getenv('DB_USER')}"
          password = “${System.getenv(‘DB_PASS')}"
    ...
      }
    }
    }

Secrets as Code

Sessions

Move sessions out of container

Demo!

session/springsession

Logger config

logback

Allow changes to config to be picked up

  • scan('30 seconds')

log actuator

  • Rest API to change logging config
    • Grails 3.3.x

Details: https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-endpoints.html

Demo!

logging/logapi

Database Changes

Database Migrations Plugin

  • d-b-m - helpful tool, but review all changes.

Flyway

  • Alternative to d-b-m
  • Uses version sql
  • Can easily run standalone

Backwards compatible DB schema

  • Add new
  • Use new
  • Remove old

  • Utilize GORM getter/setters

    String getPhoneNumber() {
    phoneNumbers.first()
    }

Demo

db/dbm
db/flyway

Infrastructure Testing

You wouldn’t make an app change without testing it.

Lint & Unit

  • Fail your CI pipelines early before you spin up a VM
  • ShellCheck && BATS (if you still love bash)
  • Foodcritic/Rubocop
  • Chef Spec

Integration

(You’re already doing this, codify it!)

$ ps aux | grep grails-app

vs.

describe service('grails-app') do
  it { should be_running }
end

serverspec

Can run inside of your Chef run

We love Groovy though!

  • Just a DSL
  • Spock + sshoogar
  • sshoogar + gradle
  • Runnnable jar

Tons of options, use tools you’re comfortable with and get value from.

Build a test anytime deploy fails

  • Service X didn’t start? Test it.

Other ideas on what to test

describe service('nginx') do
  it { should be_enabled }
  it { should be_running }
end

describe port(80) do
  it { should be_listening }
end

describe command("/usr/bin/nginx -t") do
  it "nginx syntax ok" do
    expect(subject.exit_status).to eq 0
  end
end

describe file('/opt/grailsapp/config.groovy') do
  it { should be_file }
  it { should be_owned_by 'grailsapp' }
end

Operating in Production

Centralized Logging

Demo!

SaaS - papertrail

Centralized Stats

  • Allow developers and operations to add metrics easily (statsd mentality)
  • Measure everything, graph many, alert actionable

Influxdb + Grafana + collectd

Demo!

DataDog

Sentry Exception Logging

  • Log all log.error and exceptions w/ request
  • Send them to Slack, create GH issues
  • Ties into Grails with grails-sentry plugin
  • Plugins for everything
    (log4j, logback, js frontend, ruby, python, …)

Demo

sentry.io

Logs vs. Metrics

  • Logs have lots of detail but must parse out stats
  • Metrics reduce noise, but less detail
  • Have both - cut down logs where metrics suffice

Things you forget to monitor

  • dns
  • https expire

Make it a service

Grails can build an fully executable jar

$ ln -s /var/myapp/myapp.jar /etc/init.d/myapp
$ service myapp status

Deployment

Match your lower envs

  • Docker/Vagrant/SaaS - easy
  • Others - …

Blue/Green

  • Deploy a new instance
  • Verify instance can serve traffic
  • Switch to new instance
  • Requires
    • external session-store
    • configurable load balancer
  • Warm up your new env
    • JVMs can serve the first few req slowly

Chat Ops

  • Keep team in the know when deployments are happening
  • Who started the deploy workflow, when
  • /commands for on stats
  • 0 Exceptions in production

Stop writing bash scripts

Why?

  • Hard to test
  • Less known on team
  • Obtuse syntax for complex
  • JSON? ha

Groovy CLI Tasks

  • Good CLI arg parsers
  • Very testable
  • Understood by entire team
  • Can parse JSON

Demo

  • Simple Trait based framework with tests
  • Shows common tasks & tests for them

g3summit/admin-script

Takeaway

  • Use Groovy where it makes sense/adds value
  • Use bash when it’s easier
    • psql, etc

Summary

  • Everything is code (lint-able, testable, shareable)
  • Keep adding automation as you go
  • Be more confident in changes to app and infra
  • Deliver software more quickly