Continuous Delivery to SAP HANA Cloud Platform, us...

former_member183239 · ‎01-01-2016

In this blog post I would like to demonstrate how you can deploy to HANA Cloud Platform in an automated way and with confidence using the leading source code management and collaboration platform “GitHub” and the continuous-delivery-as-service platform “Travis CI”.

Hereby I would like to first explain the software products and services you will touch and show you how to set up a working continuous delivery setup in no time – batteries included.

Why you (really) should stop deploying to production using your IDE

Deploying cloud services through the IDE is easy and great for learning purpose. But staying in the comfort zone of manually executing unit tests, integration tests and deployments in the scope of the IDE will give you a hard time reproducing builds and deployments of work increments. Especially when working in a team, it is hard to keep track of changes and keep a live site. At SAP, our top priority for delivering first-class cloud services is the principle of “live site first”. If the service breaks all resources need to be bundled to get the service up running again.

Manual and non-reproducible deployments have the risk of not knowing how the service will behave after the deployment. The smallest mistake can cause services to break and that is what you will avoid after reading this blog post.

On the other side, manual deployments are not efficient in terms of time and human resources. A manual deployment always needs to be attended and blocks the developer from developing the next work increment. Of course, a continuous integration server such as Jenkins or Travis CI might make you wait longer and give you some initial time overhead if you don’t have a large project with many quality insurance tasks.

But: Typically, human work only scales linear – automated workflows scale exponential. In addition, it is a great feeling to have an angel in the background notifying that your commit is now live and rocking!

First things first – the moving parts/prequesites

Required skills/experience

Experience with working with Git and GitHub
Experience with developing Java Web/Java EE applications
Basic experience with writing Unix/Linux bash scripts
Have a basic idea of continuous integration, deployment and delivery
No HANA skills required

SAP HANA Cloud Platform

Of course, in this tutorial I will show you how deploy with confidence to the HANA Cloud Platform, which is the PaaS of choice when running applications using the power of HANA. In this tutorial, we will use a simple Java Web application running on Tomcat. For simplicity reasons, I do not show how to connect to HANA as this tutorial is just about automating application deployments. If you want to learn more about using HANA with your HANA Cloud Platform application, please refer to SAP HANA Cloud Platform - Persistence Service

For this tutorial, you may use a trial account on HANA Cloud Platform to get yourself up to speed https://account.hanatrial.ondemand.com .

GitHub.com/GitHub Enterprise

GitHub.com is the leading source code management and collaboration platform in the cloud. If you are using an open source library chances are pretty high that it is developed and released through GitHub.com. Furthermore, many cloud services are developed and deployed from GitHub as well. If your company happens to have a GitHub Enterprise (https://enterprise.github.com/home) on-premise installation behind your firewall, you can follow this tutorial as well. If you you use GitHub Enterprise, make sure you also have a Travis CI Enterprise installation available.

You can either use a public repository or a private repository for this tutorial

Travis CI / Travis CI Enterprise

Travis CI is a continuous-integration/continuous-delivery-as-a-service platform. If you are not familiar with continuous delivery, this article by Martin Fowler might be are great read for you: Continuous Delivery

Travis CI provides you a first-class integration with GitHub Enterprise, especially with building pull requests and merge commits. It also sets a build status for every commit of your repository. Travis CI follows the principle of isolated, discardable and reproducible build environments. Your testing and deployment workflow is described by a simple text file.

You can find more information about Travis CI in the official documentation: Travis CI User Documentation

The source code

For this tutorial, I don't want to dive into details on how to develop a Java Web application on HCP. In this tutorial, I will fork the repository SAP/cloud-basecamp on GitHub.

I will demonstrate everything required to everything get up and running, but you can also easily fork my repository (MitchK/cloud-basecamp · GitHub) if you don't want to set everything up yourself. If you have questions regarding the code, don’t hesitate to leave me an issue.

Why not XSJS?

With XSJS, continuous deployments to HCP are also possible, as well but require a lot more workarounds I don’t want to focus on here. You have to consider that the current lifecycle management around XSJS is designed to deploy code manually and has its own version control system.

Update: HANA SPS11 introduces HANA XS Advanced (XSA), the successor of HANA XS. XSA will, besides many other cool updates, make XSJS code use Node.js as the default server-side JavaScript framework. With XSA, you will be able to deploy XSJS code using CI systems like, but not limited to, Travis CI. I might write an additional blog post about this. This functionality is currently (as of Feb 2016) not available on HANA Cloud Platform yet.

Why not Node.js?

Node.js is not supported by HANA Cloud Platform so far, even though I would love to show you how to set up Continuous Delivery with Node.js. SAP made the commitment to Node.js as a server-side JavaScript engine, so we can be excited what the future brings.

Why Java?

Currently, Java is the only option to demonstrate this. Once XSA becomes available on HCP, you will be able to use a lot of other technologies, such as Python, RoR, C/C++, Node.js, ...

Java JDK

You need to have a JDK installed. I recommend to have at least JDK 7 installed.

Maven

Maven is a command line tool for managing the build life cycle of Java applications. Maven can manage your dependencies, run tests, generate configurations and even deploy. In this tutorial, we mainly use the dependency management of Maven and ignore testing. Travis CI's Java build environment has a Maven installation pre-installed, so this will be a home run if you have worked a lot with Maven already.If you haven't worked with Maven before, don't worry - we are using a minimal setup.

If you are behind a firewall, please follow this guide https://maven.apache.org/guides/mini/guide-proxies.html

Neo Java Web SDK 2

The Neo Java Web SDK 2 comes with a Tomcat server run time that behaves exactly like the cloud runtime for local deployment and a command line tool, called neo.bat/neo.sh. The SDK can be downloaded and installed manually, but we will manage the SDK as a dependency using Maven so we can make sure we always have it there when we need it.

Eclipse

For this demo, I am using the Eclipse Java EE IDE with the HANA Cloud Platform tools installed. If you are using HANA Development Studio, make sure you have the HCP tools installed as well. Please refer to this link for further information: SAP Development Tools for Eclipse

You can of course also develop your application with your IDE of choice or don't use any IDE and just use plain Maven. Totally your choice.

Creating a project

Go ahead and fork the SAP/cloud-basecamp · GitHub repository on GitHub. This will copy the source code to your account or organization on GitHub.

Go to Travis-CI.org, log in with your GitHub account and connect your GitHub repository you have just created.

On your local device, go ahead and clone your project. I recommend to clone the repository via command line. If you get a permission error, you don't have your SSH keys set up correctly with GitHub. Please refer to Generating SSH keys - User Documentation for more information.



$ git clone git@github.com:your/repo.git

After that, open Eclipse and navigate to File -> Import -> Existing Maven Project. Open the directory of the repository you have just cloned.

Once the project is imported, make sure that you don't get any errors in Eclipse. If there are any, please resolve them first.

Open your pom.xml, scroll down to the <build> element. Within the <plugins> element, add the following XML snippet to add the Neo SDK as a dependency of your Maven project.

Note: The pom.xml from the base repository might be outdated in terms of package versions which most likely will cause you problems. I recommend to just copy the pom.xml from my repository: cloud-basecamp/pom.xml at master · MitchK/cloud-basecamp · GitHub



<plugin>

  <groupId>com.sap.cloud</groupId>

  <artifactId>neo-java-web-maven-plugin</artifactId>

  <version>2.38.6</version>

  <executions>

    <execution>

<phase>install</phase>

<goals>

  <goal>install-sdk</goal>

</goals>

    </execution>

  </executions>

  <configuration>

    <!-- Location of the SAP HANA Cloud Platform SDK -->

    <sdkInstallPath>${project.build.directory}/sdk</sdkInstallPath>

  </configuration>

</plugin>

Save the file, right-click on your project, select "Run As" and finally select "maven install". If you are behind a firewall and get errors like "Unable to transfer artifact", please set up your company's HTTP proxy as described here: https://maven.apache.org/guides/mini/guide-proxies.html

Now, Maven will download the SDK for you. After that, make sure the folder target/sdk exists. If it does not exist, try to refresh the project hierarchy.

Run the application locally (optional)

Now we want to test the application locally to make sure everything works. If you are confident enough, you can skip this step.

First, you need to create a server in Eclipse. We will make Eclipse point to the SDK you have just downloaded. You can also point to a SDK that you have downloaded manually and placed in another location.

Navigate to the "servers" tab and click on "Create server".

In the wizard, select SAP -> Java Web SDK Tomcat 7 Server.

On the next screen, add your web application to Tomcat. You may get errors that prevent you from adding the project to the Tomcat server. This means that your project facets of your Eclipse project may not be set up correctly. You can configure them with a right-click on your project -> Properties -> Project Facets.

After you have successfully added the Tomcat server from the Neo SDK and also added the web app to the Tomcat server, you can start the Tomcat server using Eclipse and open http://localhost:8080/basecamp/ in your browser to verify that your app works as expected.

Setting up Travis CI

Next, we will configure the integration and deployment workflow for Travis CI. With Travis CI, content-specific configuration is configured in a .travis.yml file that you put in the root directory of your repository.

In a staging/production scenario you would typically want to have multiple applications that represent the latest commit of a Git branch. For example, a production environment can represent the master branch and the staging environment can represent a dev branch. In this tutorial, I will show you how to do a deployment with a single branch and a single instance. This can by extended and scaled as you want and need it.

.travis.yml configuration

Create a file named .travis.yml (don't forget the dot) with the following content:



language: java

jdk:

- openjdk7

after_script:

  - if [[ $TRAVIS_PULL_REQUEST == 'false' && $TRAVIS_BRANCH == 'master' ]]; then bash .travis/push production || exit $?; fi

notifications:

  email: true

This YAML file will make Travis choose a Java build environment with common Java build tools installed, such as Maven. With Travis CI, you can also do cross-testing against multiple JDK versions. In this case, we just use JDK 7.

As the .travis.yml configures a Java project, Travis CI knows that a command line call of mvn install might be required. Therefore, Travis CI executes it for you. If you want to configure other behaviour, can can still do so. Please refer to the Travis CI User Documentation for more information.

For deployment, we will use a .travis/push bash script.You can name and place the files in your repository as you want. In this case, I chose to deploy this way. All files referenced in this YAML file are resolved.

Let's take a closer look at this line:



 if [[ $TRAVIS_PULL_REQUEST == 'false' && $TRAVIS_BRANCH == 'master' ]]; then bash .travis/push production || exit $?; fi

Travis provides environment variables so the workflow can be controlled by a context. In this case, we only deploy if this build was not triggered by a pull request and the branch of the commit is master. If both conditions apply, the WAR file will be deployed to production. If the deployment fails, the script will return the error code and will make the Travis CI build fail as well.

.travis/push

This script is the actual deployment script. It will upload the compiled WAR file for you and restart the server for you.



#!/bin/bash

APP_NAME=$1

NEO=./target/sdk/tools/neo.sh

pwd

echo Deploying "$APP_NAME"

$NEO deploy -a "$ACCOUNT" \

       -b "$APP_NAME" \

       -h "$LANDSCAPE_HOST" \

       -u "$DEPLOY_USERNAME" \

       -p "$DEPLOY_PASSWORD" \

       -s ./target/basecamp-1.0.0.war

echo Restarting app...

$NEO restart -a "$ACCOUNT" \

             -b "$APP_NAME" \

             -h "$LANDSCAPE_HOST" \

             -u "$DEPLOY_USERNAME" \

             -p "$DEPLOY_PASSWORD"

echo DEPLOYMENT FINISHED

As you can see, the script uses a lot of environment variables:

APP_NAME - first argument of the bash script and can be set by the call of the script in the .travis.yml
ACCOUNT - Account name on the HANA Cloud Platform, set by the Travis CI environment later
LANDSCAPE_HOST - hanatrial.ondemand.com, for example, set by the Travis CI environment later
DEPLOY_USERNAME - Your username, set by the Travis CI environment later
DEPLOY_PASSWORD - Your password, set by the Travis CI environment later

These environment variables can be either injected from outside the build using Travis CI or can be encrypted and put into the content. Travis CI provides functionality for two-factor encryption/decryption of secrets.

Note: NEVER put clear text secrets in your repository unless you want to see the world burn.

Set up environment variables and other non-content settings

Open Travis CI in your browser and navigate to your repository.Click on "Settings"

Set the concurrent builds to "1". This is important for deployments as Travis CI jobs are executed in parallel, by default. You don't want to have one deployment block another.

Furthermore, add the following environment variables:

ACCOUNT - Account name on the HANA Cloud Platform
LANDSCAPE_HOST - hanatrial.ondemand.com, for example.
DEPLOY_USERNAME - Your HCP username
DEPLOY_PASSWORD - Your HCP password

Note: Don't let your secrets show up in you build log.

The settings should now look like the following:

Commit & Push - The moment of truth

Commit your repository work and push it to master:



$ git add -A

$ git commit -m "YOLO"

$ git push origin master

After waiting a couple of seconds (Non-paying Travis CI users have to wait a couple of seconds), Travis CI will build and deploy your code.

The application on HCP might take a couple of seconds to restart. In a live environment, you might consider canary releasing and traffic routing to achieve zero downtime.

Navigate your browser to your application URL:

Conclusing and further information

Travis CI and HANA Cloud Platform can work well together. Of course, this way of deploying can be achieve with a couple of other CI products, such as Jenkins or Circle CI. Travis CI has the clear advantage over Jenkins to provision a clean environment for you and allows you to build your code with minimum configuration overhead. Travis CI also comes with excellent GitHub support out of the box.

Working example

You find the working example code here on GitHub: https://github.com/MitchK/cloud-basecamp

Please note that it was forked from SAP/cloud-basecamp · GitHub which was created by Matt Steiner (Thank you very much at this point).

Integration testing

You might want to also do integration testing, which also works well with Travis CI. Just open a database tunnel in the background, start the Tomcat server from the Neo SDK and run your test scenarios.

Multiple test instances

You can basically create a running application for every branch you have. You can also run multiple commit versions on different HCP applications. A pattern that is also seen very often is to create a unique application instance for every pull request, so you can immediately see whether your code you are proposing really runs.

Blue-Green deployment

In order to avoid downtime, you can also consider having multiple instances for production and keep one instance running as long as the new code is deploying on the other. Once the new work increment runs, you can make your domain or router switch the related instance.