Thursday 14 April 2016

How to create a SOA 12.2.1 docker image on OracleLinux


Introduction

What is docker?

Unless you’ve been living without internet access for the last two years, it would be hard not to at least heard of Docker. But, as an emerging technology not everyone has taken the time to work out what Docker is, where it fits in and how it can benefit you.

So, what exactly is Docker? Here’s what Docker themselves describe it as:

    Docker is an open platform for developers and sysadmins of distributed applications.

Essentially, Docker is a container based system for your applications. If you’re used to the concept of virtual servers, Docker provides further levels of abstraction for your application. Here’s a visual representation of how it differs:

VM vs Containers - Docker


Rather than just being one part of the puzzle, Docker provides a number of components and tools to assist with the entire lifecycle management of the application. This includes the container environment, image management and orchestration.

Docker started it’s life as an internal project within a hosting company called dotCloud, but quickly took off once they open sourced it in early 2013. Since then, it's benefited from over 15,000 software commits from over 900 contributors.

Why use Docker?

Now that you have a basic understanding of Docker, there are a number of great reasons to start using it.
  • It’s very fast. Start a Docker container can be complete in as little as 50ms. That’s not a typo, it really can be this quick! This is the advantage of having such high levels of abstraction, you reduce the number of components you need to run. This also means that there's very little to no overhead in it's implementation.
  • One command deployments . It really is as simple as installing an application with one line. Want to install MySQL? One command. Want WordPress, MySQL, Nginx and Memcache all installed and configured? Yep, it’s one command.
  • Pre-configured Apps. At last count, there were over 13,000 applications already packaged as a Docker image. Chances are, if you’re using a common application then most of the initial work has already been done for you. But, it doesn't end there. You can take the existing image, make your own changes and push it to your own repository for ease of re-deployment.
  • Resource Isolation. Previously, if you ran all of your services on the one server then there’s a chance one of them could exhaust all of the server resources. Docker allows you to set, monitor and adjust these on a per application or service basis.
  • Consistency. Docker really is the “write once deploy anywhere” type environment. It removes all of the hassles going from a development to a production environment or similar. Each set of libraries is very tightly coupled to the docker image to ensure the consistency.
  • A complete platform. Rather than just being one part of the puzzle, Docker is shaping up to be a complete platform. There’s the base Engine for the containers, the Registry for image management, Compose to orchestrate complex deployments, Swarm for clustering and Machine for provisioning. This is what’s made Docker different from other container implementations, you can manage the entire lifecycle quite easily.
  • Scale. This is is one area where Docker really shines, especially if you have a micro-service based application. Compose and Swarm assist with deploying scalable applications and there’s then third party tools like Kubernetes and Mesos, which both take it to the next level. We’re talking the ability to manage the entire lifecycle with up to millions of containers, so scale isn’t a problem!
Moreover Oracle is now quite focused on certifying its Oracle Fusion Middleware for Docker. Details are here. Also below is the current status of certification Docker related:

Oracle Linux x64 Oracle Linux 6(UL5+)
Oracle Linux 7(UL0+)
Red Hat Enterprise Linux 7(UL0+)
Docker Containers
  • Oracle WebLogic Server 12c (12.2.1) (Oracle Linux 6 (UL6+) is required. Refer to Note 6)
  • Oracle WebLogic Server 12c (12.1.3) (Refer to Note 4)


How can docker be useful to us?

Docker could  be  particularly useful for the following reasons;
  • More reliable releases
  • Enable continuous delivery
  • Recreate environments
  • Internal tools

Docker Installation

This guide below assumes the operating system is Oracle Linux 7.
Oracle provides a very good guide available here on how to install and configure Docker on OL7 (it is also available for OL6). I will just post here the main steps.

Operating systems pre-requirement

Docker version 1.9 and later require that you configure the system to use the Unbreakable Enterprise Kernel Release 4 (UEK R4) and boot the system with this kernel.

  • Disable access to the ol7_x86_64_UEKR3 channel and enable access to the ol7_x86_64_UEKR4 channel.

If you use Oracle Public Yum, disable the ol7_UEKR3 repository and enable the ol7_UEKR4 repository in the /etc/yum.repos.d/public-yum-ol7.repo file, for example:

[ol7_UEKR3]
name=Latest Unbreakable Enterprise Kernel Release 3 for Oracle Linux $releasever ($basearch)
baseurl=http://public-yum-qa.oracle.com/repo/OracleLinux/OL7/UEKR3/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=0

[ol7_UEKR4]
name=Latest Unbreakable Enterprise Kernel Release 4 for Oracle Linux $releasever ($basearch)
baseurl=http://public-yum-qa.oracle.com/repo/OracleLinux/OL7/UEKR4/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1

You can download the latest version of this file from http://public-yum.oracle.com/public-yum-ol7.repo.

  • Run the following command to upgrade the system to UEK R4:

# yum update

It is also recommended to make UEK R4 the default boot kernel, see Section 4.3, “About the GRUB 2 Boot Loader” at https://docs.oracle.com/cd/E52668_01/E54669/html/ol7-grub2_bootloader.html.

  • Reboot the system, selecting the UEK R4 kernel (version 4.1.12) if this is not the default boot kernel.

# systemctl reboot

  • Enable the ol7_addons repository in the /etc/yum.repos.d/public-yum-ol7.repo file, for example:

[ol7_addons]
name=Oracle Linux $releasever Add ons ($basearch)
baseurl=http://public-yum.oracle.com/repo/OracleLinux/OL7/addons/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1

Docker engine Installation

  • Install the docker-engine package.

# yum install docker-engine

By default, the Docker Engine uses the device mapper as a storage driver to manage Docker containers. 
As with LXC, there are benefits to using the snapshot features of btrfs instead.

  • Configure Docker images to be saved into btrfs  filesystem (optional but strongly recommended)
Oracle recommends using btrfs because of the stability and maturity of the technology. If a new device for btrfs is not available, you should use overlay as the storage driver instead of device-mapper for performance reasons. You can configure overlay by adding the --storage-driver=overlay option to DOCKER_STORAGE_OPTIONS in /etc/sysconfig/docker-storage. The overlayfs file system is available with UEK R4.
For more information, see https://docs.docker.com/engine/userguide/storagedriver/overlayfs-driver/.

To configure the Docker Engine to use btrfs instead of the device mapper:

Use yum to install the btrfs-progs package.

# yum install btrfs-progs

If the root file system is not configured as a btrfs file system, create a btrfs file system on a suitable device such as /dev/sdb in this example:

In the case you are using Virtual Box (or any virtual machines tool) this is quite easy, just follow the steps below (details are available here):

1. Add Disk Storage to Oracle Virtual Box using the tool (make sure it's big! I used a 50 GB one)

2. Turn on the VirtualBox OL7 machine and switch to the root user.

3. Then use fdisk utility to get list of hard drives.
$ fdisk -l

Disk /dev/sda: 107.4 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000e4833

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          64      512000   83  Linux Partition 1 does not end on cylinder boundary.
/dev/sda2              64       13055   104344576   8e  Linux LVirtualBox

Disk /dev/sdb: 12.9 GB, 12884901888 bytes
255 heads, 63 sectors/track, 1566 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/mapper/VolGroup-lv_root: 53.7 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

.....

From the first block you can clearly see that we have two hard drives i.e. /dev/sda and /dev/sdb. 
You can also see that /dev/sda has been formatted and mounted using logical volume management. 
This was surely done when you created the VirtualBox or when the template was created, as it is the default way Linux formats the hard drives.

4. We need to partition the /dev/sdb device. Use the following command to partition the device.

fdisk /dev/sdb 

Follow the screen instructions as shown below.
# fdisk /dev/sdb
[/sociallocker] Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x2e3c77cd.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): n
Command action
e   extended
p   primary partition (1-4)
p

Partition number (1-4): 1
First cylinder (1-1566, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1566, default 1566):
Using default value 1566

Command (m for help): w

The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks. 
We have used ‘n’ to create a new partition, p to create the partition as primary, 1 to specify the partition number and used defaults for begin and end range for partition size.

Finally ‘w’ to make changes permanent.

5. Query the partition table again to see now that /dev/sdb1 partition has been created into /dev/sdb disk.

# fdisk -l

6. Format the Partition using the file system (btrfs) recommended

mkfs.btrfs /dev/sdb1

7. Use the blkid command to display the UUID of the device and make a note of this value, for example:

# blkid /dev/sdb1
/dev/sdb: UUID="a7dc508d-5bcc-4112-b96e-f40b19e369fe" \
  UUID_SUB="1aa666eb-0861-4dc8-a37e-f3c87c7003b8" TYPE="btrfs" 

8. Create the file /etc/systemd/system/var-lib-docker.mount with the following contents:

[Unit]
Description = Docker Image Store

[Mount]
What = UUID=REPLACE_HERE_THE_UUID_value
Where = /var/lib/docker
Type = btrfs

[Install]
WantedBy = multi-user.target


This file defines the target that systemd uses to mount the file system on /var/lib/docker.

9. Enable the var-lib-docker.mount target.

# systemctl enable var-lib-docker.mount
ln -s '/etc/systemd/system/var-lib-docker.mount' \
  '/etc/systemd/system/multi-user.target.wants/var-lib-docker.mount'

This command enables systemd to mount the file system when required for use with the docker service. It does not mount the file system. If you need to mount the file system independently of Docker, use the following command:

# systemctl start var-lib-docker.mount

Create the drop-in file /etc/systemd/system/docker.service.d/var-lib-docker-mount.conf, which contains the following lines:

[Unit]
Requires=var-lib-docker.mount
After=var-lib-docker.mount

These entries tell systemd to mount the /var/lib/docker file system by using the var-lib-docker.mount target before starting the docker service.

After doing this restart the OS and make sure Docker restart correctly.

Moreover running the command docker info you can check the file system configuration is now applied:
Containers: 0
Images: 0
Server Version: 1.9.1
Storage Driver: btrfs
 Build Version: Btrfs v3.19.1
 Library Version: 101
Execution Driver: native-0.2

Problems:

In my test, Docker did not restart, so I checked at the log using the command below
more /var/log/messages | grep docker

I saw this:
SELinux is not supported with the BTRFS graph driver!

Googling the issue I just found a workaround to this problem (not sure for PROD environment if acceptable)

Edit the file 
vi /etc/sysconfig/docker

Removing the selinux-enabled from the options as below:
OPTIONS=''
#'--selinux-enabled'

Restart Docker!
# systemctl daemon-reload
# systemctl restart docker

Docker Post-Installation steps

  • Start the docker service and configure it to start at boot time.
# systemctl start docker
    # systemctl enable docker
      ln -s '/etc/systemd/system/docker.service' \
          '/etc/systemd/system/multi-user.target.wants/docker.service'
          systemctl status docker
            Useful commands to check Docker status:

            # docker info
            # docker version
            • Enabling Non-root Users to Run Docker Commands

            Create the docker group:

            # groupadd docker
            Restart the docker service:

            # service docker restart
            The UNIX socket /var/run/docker.sock is now readable and writable by members of the docker group.

            Add the users that should have Docker access to the docker group:

            # usermod -a -G docker user1

            SOA Image Creation

            To create the Docker SOA 12.2.1 image I used this blog post here, also the DockerFile I used are available here.
            Therefore make sure you execute the following steps:
            1. open a terminal in the VM and suso as a Root, then
              1. mkdir /fmw/soa
              2. cd /fmw/soa
              3. mkdir installers
            2. Copy into installers the following files:
              1. fmw_12.2.1.0.0_infrastructure_Disk1_1of1.zip
              2. fmw_12.2.1.0.0_soaqs_Disk1_1of2.zip
              3. fmw_12.2.1.0.0_soaqs_Disk1_2of2.zip
              4. jdk-8u77-linux-x64.gz
              5. silent.rsp (download it from the blog page as above and customize)
            3. Copy into fmw/soa the following file:
              1. Dockerfile.12.2.1.0.0 (download it from the blog page as above and customize)
            4. Change directory to /fmw/soa and execute the below command:
              1. docker build -t oracle/fmw:12.2.1.0.0 -f ./Dockerfile.12.2.1.0.0 .
            5. Run the container just created with the folowing
              1. docker run -i -P --name=oracle-soa-1 -t oracle/fmw:12.2.1.0.0
            6. Accessing to the machine please verify everything has been installed successfully (you could tsart Jdev following the procedure here below at export via X11)

            Oracle XE Image Creation

            To create the Docker Oracle XE 11 image I used the Docker file here.
            Therefore make sure you execute the following steps:
            1. open a terminal in the VM and suso as a Root, then
              1. mkdir /database/
              2. cd /database/
              3. mkdir xe 
            2. Copy into xe the following files:
              1. Dockerfile11.2.0.1.0 (download it from the blog post above)
              2. oracle-xe-11.2.0-1.0.x86_64.rpm.zip 
            3. Change directory to /database and execute the below command:
              1. docker build -t oracle/db/xe:11.2.0.1.0 -f ./xe/Dockerfile.11.2.0.1.0 .
            4. Run docker
              1. docker run -i -P --name=oracle-db-xe-1 -t oracle/db/xe:11.2.0.1.0
            5. Test the Database is up and running connecting via SQL Developer o similar tools (you can connect from any place is the same subnet, check the external port by running docker ps -a)

            Possible Problems

            If you have tried to create Docker containers for Oracle Database XE, you might not be able to configure/start a database instance successfully. The installation may complete successfully but when you try to configure or start an instance, you might notice the following error in the logs.

            ORA-01034: ORACLE not available
            ORA-27101: shared memory realm does not exist
            Linux-x86_64 Error: 2: No such file or directory
            ORA-00845: MEMORY_TARGET not supported on this system

            Reason might be related to the Docker version used. Oracle Database needs a shared memory [/dev/shm] available of at least 1 GB, while any Docker container (up to 1.9 version) is 64MB.

            Some blog post suggest to get the latest of docker (10.X) and make use of the new option --shm-size=2g:

            cat >/etc/yum.repos.d/docker.repo <<-EOF
            [dockerrepo]
            name=Docker Repository
            baseurl=https://yum.dockerproject.org/repo/main/oraclelinux/7
            enabled=1
            gpgcheck=1
            gpgkey=https://yum.dockerproject.org/gpg
            EOF

            Then make sure docker engine is update running the below
            yum install docker-engine

            Other approach (like the one I used just avoid this issue adding the following in the Docker file:

            # Work around sysctl limitation of docker
            RUN sed -i -e 's/^\(memory_target=.*\)/#\1/' /u01/app/oracle/product/11.2.0/xe/config/scripts/initXETemp.ora \
                && sed -i -e 's/^\(memory_target=.*\)/#\1/' /u01/app/oracle/product/11.2.0/xe/config/scripts/init.ora

            Also creating the XE server database FROM oraclelinux:7 (6 does not work)

            SOA Domain creation

            In order to create a domain, first of all the SOA docker container needs to be able to talk with the Database one. Docker by default allows containers to see each other however it is recommended to have different network.
            Good practice is to create sub-network in the Docker engine. This can be performed with the following steps:
            1. docker network create my-soa-network
            2. Run the container just created with the following (here adding -u=0 the container will be accessed as a ROOT user, which will allow to install telnet for testing the connection to the database "yum install telnet") 
              1. docker run -i -P --net=my-soa-network --name=oracle-soa-1 -t oracle/fmw:12.2.1.0.0
            3. Run in another terminal
              1. docker run -i -P --net=my-soa-network --name=oracle-db-xe-1 -t oracle/db/xe:11.2.0.1.0
            Second step is to create the SOA Db Schemas using the RCU, to do that use ifconfig to get the DB IP, then:
            1. In the SOA Docker container creates a file "passwd.txt" with 1 password for each database user that RCU is going to create. For the SOA Domain 7 password are needed.
            2. Then execute the following:
              1. rcu -silent -createRepository -connectString <DB IP>:1521:XE -databaseType ORACLE -dbUser SYS -dbRole SYSDBA -schemaPrefix <YOUR PREFIX> -variables SOA_PROFILE_TYPE=SMALL -component SOAINFRA -component OPSS -component IAU -component IAU_APPEND -component IAU_VIEWER -component MDS -component WLS -component UCSUMS -f < passwd.txt
            Last step is to create the domain using WLST script
            1. Run the following wlst Script using you r enrionment details
              1. WLST

            Tips

            Display export via X11

            Sometimes it is nice to have the UI.
            Docker and the OracleLinux template by default does not support any UI as it supposed to be a lightweight system.
            However exporting the display it is easy and can be done with the following steps:
            1. Run the image with the option -u (user root) as below
              1.  docker run -u 0 -it --name soa1221 oracle/fmw:12.2.1.0.0
            2. Install wget and download and install the X11 rpm packege
              1. yum install wget
              2. wget http://vault.centos.org/6.2/os/x86_64/Packages/xorg-x11-server-Xvfb-1.10.4-6.el6.x86_64.rpm
              3. yum localinstall xorg-x11-server-Xvfb-1.10.4-6.el6.x86_64.rpm
            3. Change user to oracle, export the DISPLAY and run Jdev
              1. su oracle
              2. export DISPLAY=192.168.1.220:1.0   (assuming this is the target IP where you'd like to UI to appear, of course this reuqires an X11 server, in my case I used MobaXTerm, a very simple one)
              3. Run Jdeveloper or any other tool with UI

            SSH Server

            In case the SSH Serer is needed the below command can be added to the DockerFile creation, also the container needs to be started with this option -p 22

            RUN yum -y install openssh-server

            However opening the ssh communication is not recommended while it is much more secure to use the docker EXEC <container> bash command.

            8 comments:

            1. Hi Fabio,

              First of all, congratulations for the excellent post.
              I was checking the Oracle page regarding SOA Suite certification for Docker, as I thought SOA Suite was not certified, and I could not find SOA Suite in the list. Only weblogic. Can you let me know where I can get more information / confirmation about SOA Suite be Oracle Certified for Docker?

              Thanks,
              Rafael

              ReplyDelete
            2. Hi Rafael,
              the certified Oracle FM on Docker are here (as mentioned in the article): http://www.oracle.com/technetwork/middleware/ias/oracleas-supported-virtualization-089265.html#wls1221supportfordockercontainers.

              SOA Suite 12.2.1 was here and I actually just realised is not there anymore.
              I'll try to get more info and eventually let you know.

              Thanks Fabio

              ReplyDelete
              Replies
              1. Hi Rafael
                I just got confirmation that Oracle Soa 12c is not officially certified. It will be certified later for Dev and test.
                Sorry this is all I know so far.
                Regards

                Delete
            3. Amazing port, but wonder if you have tested this by yourself. because if I follow it step by step, it will not work.

              ReplyDelete
            4. I read your post when it was fresh. I stumbled on this link again while searching for something else. Since this appears to be favored by Google it is worth pointing out MS has changed its mind.

              In today's job market, Docker skills are highly appreciated. The Docker Certified Assistant (DCA) is the foundation for the actual Docker skills of the entire container industry. Are you ready to prove your Docker skills? Get Docker certification now! Docker certification is designed to reflect the needs of organizations and IT professionals.
              KillerDumps offers the entire IT exam dump file and practice test simulator at a discounted price with 100% guaranteed success. We provide ongoing after-sales support for Docker Q&A and practice software to address any inconvenience.

              You can get Docker Exam Dumps

              ReplyDelete
            5. Thanks a lot for sharing a valuable blog on Oracle SOA Training. I was browsing through the internet looking for Oracle SOA Training and came across your blog. I am impressed by the information that you have on this blog. It shows how well you understand this subject, you can find more information about Oracle SOA by attending Oracle SOA Training. You can learn about interview questions by visiting fusion Interview Questions

              ReplyDelete
            6. Exam4Lead provide you extraordinary and most relevant material to pass your exams. Your investment is like a deal that keep your money save. You can demand for 100% refund if you are not got the promised results.

              ReplyDelete