Upgrading Apigee hybrid to version 1.4

Upgrading to version 1.4.4 overview.

The procedures for upgrading Apigee hybrid are organized in the following sections:

Backup your hybrid installation.
Check your Kubernetes version and upgrade as appropriate.
Upgrade ASM.
Install hybrid runtime version 1.4.

Prerequisite

Apigee hybrid version 1.3. If you are updating from an earlier version see the instructions for Upgrading Apigee hybrid to version 1.3.

Upgrade to version 1.4

(Recommended) Make a backup copy of your version 1.3 $APIGEECTL_HOME/ directory. For example:
```
tar -czvf $APIGEECTL_HOME/../apigeectl-v1.3-backup.tar.gz $APIGEECTL_HOME
```
(Recommended) Backup your Cassandra database following the instructions in Cassandra backup and recovery

Upgrade your Kubernetes platform as follows. Follow your platform's documentation if you need help:

Platform	Upgrade to version
GKE	1.18.x
GKE on-prem (Anthos)	1.5.x
AKS	1.18.x using Anthos attached clusters
EKS	1.18.x using Anthos attached clusters

Upgrade ASM to version 1.6.x.
You may experience a brief interruption of incoming traffic (typically less than two seconds) when you upgrade ASM. If you are running hybrid in two or more clusters, you can avoid this interruption by:
1. Disable your load balancer.
2. Upgrade ASM on one cluster at a time.
3. Enable your load balancer.
1. Perform the upgrade using the ASM documentation:
  - GKE on-prem (Anthos): Upgrading Anthos Service Mesh on premises
  - All other platforms: Upgrading Anthos Service Mesh on GKE
2. Specify the following ingress ports and set the runAsRoot property to true in your istio-operator.yaml file"
```
    … … …
            ports:
            - port: 15021
              name: status-port
              targetPort: 15021
            - port: 80
              name: http2
              targetPort: 80
            - port: 443
              name: https
              targetPort: 443

    … … …
    values:
      gateways:
        istio-ingressgateway:
          runAsRoot: true 
```
3. Create PeerAuthentication CRs for disabling mTLS in your istio-operator.yaml file:
```
---
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: apigee-system
  namespace: apigee-system
spec:
  mtls:
    mode: DISABLE
---
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: user-namespace
  namespace: user-namespace
spec:
  mtls:
    mode: DISABLE
```
4. Apply these changes with istioctl as described in the ASM documentation:

Install the hybrid 1.4.4 runtime

Store the latest version number in a variable:

export VERSION=$(curl -s \
    https://storage.googleapis.com/apigee-release/hybrid/apigee-hybrid-setup/current-version.txt?ignoreCache=1)

Check that the variable was populated with a version number. If you want to use a different version, you can save that in an environment variable instead. For example:
```
echo $VERSION
```
```
  1.4.4
```

Download the release package for your operating system:

Mac 64 bit:

curl -LO \
    https://storage.googleapis.com/apigee-release/hybrid/apigee-hybrid-setup/$VERSION/apigeectl_mac_64.tar.gz

Linux 64 bit:

curl -LO \
    https://storage.googleapis.com/apigee-release/hybrid/apigee-hybrid-setup/$VERSION/apigeectl_linux_64.tar.gz

Mac 32 bit:

curl -LO \
    https://storage.googleapis.com/apigee-release/hybrid/apigee-hybrid-setup/$VERSION/apigeectl_mac_32.tar.gz

Linux 32 bit:

curl -LO \
    https://storage.googleapis.com/apigee-release/hybrid/apigee-hybrid-setup/$VERSION/apigeectl_linux_32.tar.gz

Rename your current apigeectl/ directory to a backup directory name. For example:
```
mv $APIGEECTL_HOME/ $APIGEECTL_HOME-v1.2/ 
```
Extract the downloaded gzip file contents into your hybrid base directory. For example:
```
tar xvzf filename.tar.gz -C hybrid-base-directory
```
cd to the base directory.
The tar contents are, by default, expanded into a directory with the version and platform in its name. For example: ./apigeectl_1.4.4-xxx_linux_64. Rename that directory to apigeectl:
```
mv apigeectl_1.4.5-xxx_linux_64 apigeectl
```
In the new apigeectl/ directory, run apigeectl init, apigeectl apply, and apigeectl check-ready:
1. Initialize hybrid 1.4.4:
```
apigeectl init -f overrides.yaml
```
  Where overrides.yaml is your edited overrides.yaml file.
2. Check that it initialized correctly with the following commands:
```
apigeectl check-ready -f overrides.yaml
```
```
kubectl describe apigeeds -n apigee
```
  Your output should look something like:
```
Status:
  Cassandra Data Replication:
  Cassandra Pod Ips:
    10.8.2.204
  Cassandra Ready Replicas:  1
  Components:
    Cassandra:
      Last Successfully Released Version:
        Revision:  v1-f8aa9a82b9f69613
        Version:   v1
      Replicas:
        Available:  1
        Ready:      1
        Total:      1
        Updated:    1
      State:        running
  Scaling:
    In Progress:         false
    Operation:
    Requested Replicas:  0
  State:                 running
```
3. The syntax of the apigeectl --dry-run flag depends on the version of kubectl you are running. Check the version of kubectl:
```
gcloud version
```
4. Check for errors with a dry run with the command appropriate to your version of kubectl:
  kubectl version 1.17 and older:
```
apigeectl apply -f overrides.yaml --dry-run=true
```
  kubectl version 1.18 and newer:
```
apigeectl apply -f overrides.yaml --dry-run=client
```
5. Apply your overrides. Select and follow the instructions for production environments or demo/experimental environments, depending on your installation.
  During the upgrade process each component will perform a rolling restart. Therefore, for production environments, it is best to apply the upgrade to one component at a time.
  Production
  
  For production environments you should upgrade each hybrid component individually, and check the status of the upgraded component before proceeding to the next component.
  1. Apply your overrides to upgrade Cassandra:
    apigeectl apply -f overrides.yaml --datastore
  2. Check completion:
    kubectl -n namespace get pods
    
    apigeectl check-ready -f overrides.yaml
    
    Where namespace is your Apigee hybrid namespace.
    
    Proceed to the next step only when the pods are ready.
  3. Apply your overrides to upgrade Telemetry components and check completion:
    apigeectl apply -f overrides.yaml --telemetry
    
    kubectl -n namespace get pods
    
    apigeectl check-ready -f overrides.yaml
  4. Apply your overrides to upgrade the org-level components (MART, Watcher and Apigee Connect) and check completion:
    apigeectl apply -f overrides.yaml --org
    
    kubectl -n namespace get pods
    
    apigeectl check-ready -f overrides.yaml
  5. Apply your overrides to upgrade your environments. You have two choices:
    
    Environment by environment: Apply your overrides to one environment at a time and check completion. Repeat this step for each environment:
    apigeectl apply -f overrides.yaml --env env_name
    
    kubectl -n namespace get pods
    
    apigeectl check-ready -f overrides.yaml
    
    Where env_name is the name of the environment you are upgrading.
    
    All environments at one time: Apply your overrides to all environments at once and check completion:
    apigeectl apply -f overrides.yaml --all-envs
    
    kubectl -n namespace get pods
    
    apigeectl check-ready -f overrides.yaml
  Demo/Experimental
  
  In most demo or experimental environments, you can apply the overrides to all components at once. If your demo/experimental environment large and complex or closely mimics a production environment, you may want to use the instructions for upgrading production environments
  1. apigeectl apply -f overrides.yaml
  2. Check the status:
    apigeectl check-ready -f overrides.yaml
Congratulations! You have successfully upgraded to Apigee hybrid version 1.4.4.