Submit social housing lettings and sales data (CORE)

core

6.6 KiB

Raw Blame History

nav_order
5

Infrastructure

Configuration

On GOV.UK PaaS, service credentials are appended to the environment variable VCAP_SERVICES when services are bound to an application. Such services include datastores and S3 buckets.

Our application uses S3 and Redis clients and supports two different ways of parsing their configuration:

Via the environment variable VCAP_SERVICES using the PaasConfigurationService class
Via the environment variables S3_CONFIG and REDIS_CONFIG using the EnvConfigurationService class

S3_CONFIG and REDIS_CONFIG are populated using a similar structure than VCAP_SERVICES:

S3_CONFIG:

[
  {
    "instance_name": "bucket_1",
    "credentials": {
      "aws_access_key_id": "123",
      "aws_secret_access_key": "456",
      "aws_region": "eu-west-1",
      "bucket_name": "my-bucket"
    }
  }
]

REDIS_CONFIG:

[
  {
    "instance_name": "redis_1",
    "credentials": {
      "uri": "redis_uri"
    }
  }
]

In order to switch from using GOV.UK PaaS provided services to external ones, instances of PaasConfigurationService need to be replaced by EnvConfigurationService. This assumes that S3_CONFIG or/and REDIS_CONFIG are available.

Please check full_import.rake and rack_attack.rb for examples of how the configuration is used.

Deployment

This application is running on GOV.UK PaaS. To deploy you need to:

Contact your organisation manager to get an account in dluhc-core organization and in the relevant spaces (staging/production).
Install the Cloud Foundry CLI

cf login -a api.london.cloud.service.gov.uk -u <your_username>

Set your deployment target (staging/production):

cf target -o dluhc-core -s <deploy_environment>

Deploy:
```
cf push dluhc-core --strategy rolling
```
This will use the manifest file

Once the app is deployed:

Get a Rails console:

cf ssh dluhc-core-staging -t -c "/tmp/lifecycle/launcher /home/vcap/app 'rails console' ''"

Check logs:
```
cf logs dluhc-core-staging --recent
```

Troubleshooting deployments

A failed Github deployment action will occasionally leave a Cloud Foundry deployment in a broken state. As a result all subsequent Github deployment actions will also fail with the message Cannot update this process while a deployment is in flight.

cf cancel-deployment dluhc-core

You would then need to check the logs and fix the issue that caused the initial deployment to fail.

CI/CD

When a commit is made to main the following GitHub action jobs are triggered:

Test: RSpec runs our test suite
Deploy: If the Test stage passes, this job will deploy the app to our GOV.UK PaaS account using the Cloud Foundry CLI

When a pull request is opened to main only the Test stage runs.

Setting up Infrastructure for a new environment

Staging

cf login -a api.london.cloud.service.gov.uk -u <your_username>

Set your deployment target (staging):
```
cf target -o dluhc-core -s staging
```

Create required Postgres, Redis and S3 bucket backing services (this will take ~15 mins to finish creating):

cf create-service postgres tiny-unencrypted-13 dluhc-core-staging-postgres
cf create-service redis micro-6.x dluhc-core-staging-redis
cf create-service aws-s3-bucket default dluhc-core-staging-csv-bucket
cf create-service aws-s3-bucket default dluhc-core-staging-import-bucket
cf create-service aws-s3-bucket default dluhc-core-staging-export-bucket

Deploy manifest:

cf push dluhc-core-staging --strategy rolling

Bind S3 services to app:

cf bind-service dluhc-core-staging dluhc-core-staging-csv-bucket
cf bind-service dluhc-core-staging dluhc-core-staging-redis
cf bind-service dluhc-core-staging dluhc-core-staging-import-bucket -c '{"permissions": "read-only"}'
cf bind-service dluhc-core-staging dluhc-core-staging-export-bucket -c '{"permissions": "read-write"}'

Create a service keys for accessing the S3 bucket from outside Gov PaaS:

cf create-service-key dluhc-core-staging-csv-bucket csv-bucket -c '{"allow_external_access": true}'
cf create-service-key dluhc-core-staging-import-bucket data-import -c '{"allow_external_access": true}'
cf create-service-key dluhc-core-staging-export-bucket data-export -c '{"allow_external_access": true, "permissions": "read-only"}'

Production

cf login -a api.london.cloud.service.gov.uk -u <your_username>

Set your deployment target (production):
```
cf target -o dluhc-core -s production
```