* Add missing addresses csv job
* Update missing addresses csv service methods
* Add rake task
* Update the job to send missing town or city templates
* Update service to create mising town or city templates
* Add send missing town or city csv rake task
* Add IDs to the CSVs
* Put all log in the same csv
* Add issue type column
* Write wrong uprn logs to csv
* Add mailer methods
* Skip uprn issue for specified orgs
* Add sales csv rake task
* set SKIP_UPRN_ISSUE_ORG_IDS on review apps
* test
* Update notify template IDs for testing
* Initialize service with organisation instead of a hash
* Add expiration time to url
* Add optional tags and remove LA from csv
* Extract log to csv methods
* Update casing
* Update old IDs in factories
* Move constant
* Extract some repeating scopes
* Pass in organisations to skip instead of using an env var
* update template id for sales
* Update link expiry time and headers
* Lower the threshold for testing
* Add issue explanation to the email
* Add how to fix
* update emails
* CLDC-2810 Create all addresses CSV (#1953)
* Add rake tasks for creating all addresses CSV
* Write headers if logs don't exist, update header names
* Rename method
* CLDC-2810 Correct addresses from csv (#1957)
* Updating importing lettings addresses form csv
* Add import_sales_addresses_from_csv rake
* Allow correcting addresses from both templates
* escape .
* Reinfer LA if the postcode hasn't changed
* Update labels and email content
* Update missing addresses threshold
* Remove unused env var
* Adjust xml export to export refused details, update refused to take into account details_unknown
* Recalculate refused values for lettings logs with unknown person details
* Export refused values in the csv
* Update file names
* Move single person details if person 2 details are missing
* Allow reimporting details for any single missing person
* Reimport multiple missing person details, skip logs with no extra person details
* Skip update if record does not have hhmemb value
* Refactor
* Rebase change
* Extract method
* Confirm scharge, pscharge and supcharge soft validations for completed logs
* This test seems flaky, likely because of the order, so reducing #of logs
* Add generate_missing_answers_report task and method
* Create example report
* change task description
* refactor
* Report old form id instead of old id
* Make has_other_client_group a mandatory question
* Add a rake task to update has_other_client_group
* test and lint
* Set has_other_client group on import
* feat: update application to use non paas and paas configuration services, add helper to determine if platform is paas
feat: don't double-instantiate the configuration service
feat: add self. to is_paas? definition, and require module in rack_attack
chore: lint
* test: update suite to work with new paas/env config service logic
test: new PlatformHelper
* fix: update full import tests to check paas and non-paas config services
* fix: update test descriptions and remove unrequired line
---------
Co-authored-by: Sam Seed <sam.seed@softwire.com>
* Add import report service
* Call ImportReportService from rake task
* Move generate_logs_report to import service
* Add organisations without active data coordinators to the report
* pluralize method names
* update suffixes
* Remove new_data_protection_confirmation? flag
* Update schema
- drop index
- add org and user data to dpc table
* Persist org and user data at import
* Do not show invalid emails
* Persist user and org data while signing agreement
* Add migration to persist org and user data on DPC
* Rebase fix
* fix typo
* Add test as a proof of the bug we have in prod
* Refactor script to create users with unique email
* Add rake task to import missing data protection confirmations
* Add local authority links
* Display all local authorities for location
* set correct LA for log based on year
* Format local authorities for locations
* Rename variable
* Add import task for la links
* look at form start date because log start date might not be given
* Update app/models/lettings_log.rb
Co-authored-by: James Rose <james@jbpr.net>
* Update app/helpers/locations_helper.rb
Co-authored-by: James Rose <james@jbpr.net>
* Refactor
* Seed review app
* Change dates format
* Update the local authority link data
* Typo
* update mapping in Lettings logs
---------
Co-authored-by: James Rose <james@jbpr.net>
* Add local authorities table
* Seed initial local authorities
* set previous_location_only to false by default
* Add unique index and rake task
* Update field name and required fields
* Remove previous_location_only field
* tests
* Only process sales logs in sales import job
* Only process lettings logs in lettings import job
* Add sales log import task to the full import
* change review app import bucket name
* Update mortgageused and joinmore
* Update jointmore and ownership scheme
* Fix ownershipsch so that it doesn't override to nil
* Set default relat2, update default household count
* het ownership from type if not given
* Improve logging
* Remove mortgageused method
* Look at Q16aProplensec2 column if Q16aProplen2 is empty
* Set default income used and pregblank
* Remove fields calculated internally
* Comment out sales import from full import task
* typo and change bucket name
* Create test fixtures
* Add old id column
* Add sales logs importer
* Save and update completed discounted ownership log without postcode
* Update fixtures to be incomplete
* Import a completed shared ownership example
* Test with a non homebuy shared ownership example
* Add privacynotice
* Update hholdcount and confirmed fields
* Add buyer stilll serving mapping
* Add totadult/totchild and outright sale examples
* Update log fixtures
* Extract shared methods into logs import service and lint
* Add sales logs import rake task
* Update noint, xml examples and some mappings
* Add tests for checking that all required questions are answered
* Update tests, clean up import
* Map mortgage lender and mortgage lender other
* Infer Mscharge known as no for outright sale
* refactor setting default values
* when the armedforcesspouse is not answered set is as don't know
* Refactor tests: change log id names
* set savings to not known if not given
* Refactor tests: change nesting
* Backfill default household characteristics for completed log
* Add more default mapping
* Typo
* Improve logging and refactor tests
* Adjust test data to fit with the mappings that are known so far
* Rename fixture files
* Schedule data export tasks in sidekiq-cron
- Split combined XML and CSV data jobs into seperate background jobs
- Add sidekiq-cron to the project
- Schedule new XML and CSV jobs to run every day at 5am
- Adjust data export Rake tasks to use new job
* Add specs for DataExport{Xml,Csv}Job
* x
* Run bundle install
* Change review app export bucket name
---------
Co-authored-by: Kat <katrina@kosiak.co.uk>
kosiakkatrina
Jack
natdeanlewissoftwire
Chirag-Bhatti
Rachael Booth
James Rose