* CLDC-3248: Ensure previous_la_known gets cleared if prevloc is when blanking invalid fields
* CLDC-3248: Update tests around blank_invalid_non_setup_fields
* Add new nationality columns
* Add nationality questions to lettings
* Add nationality questions to sales
* Infer nationality where appropriate
* Add synonyms
* Update csv exports
* Export nationality_all for 24/25
* Update prefers not to say value
* Set 24/25 nationality fields from BU
* Undo the export (there's a separate ticket)
* Correctly display answers
* Rebase fixes, tests and lint
* Set nationality from lettings BU 24/25
* Rename files, add synonyms and accents
* Fix sales csvs
* Extract constant
* Correctly route to sales owning organisation page
* Display correct owning organisation options
* Set managing organisation as user organisation
* Add managing organisation question for sales
* Display managing organisation users in created_by
* set managing org to be same as owning org in BU
* Add managed by filter to sales
* Display reported by, correctly update owning org
* Update factory and 2022 BU
* refactor
* Merge managed sales logs
* test
* Update managing organisation for sales bulk upload
* Rebase tests
* skip a test as it is not relevant as the rake task would have already been ran
* PO fixes
* Remove old collection managed sales validation
* Update question header and filters conditions
* remove feature toggle
* Add missing addresses csv job
* Update missing addresses csv service methods
* Add rake task
* Update the job to send missing town or city templates
* Update service to create mising town or city templates
* Add send missing town or city csv rake task
* Add IDs to the CSVs
* Put all log in the same csv
* Add issue type column
* Write wrong uprn logs to csv
* Add mailer methods
* Skip uprn issue for specified orgs
* Add sales csv rake task
* set SKIP_UPRN_ISSUE_ORG_IDS on review apps
* test
* Update notify template IDs for testing
* Initialize service with organisation instead of a hash
* Add expiration time to url
* Add optional tags and remove LA from csv
* Extract log to csv methods
* Update casing
* Update old IDs in factories
* Move constant
* Extract some repeating scopes
* Pass in organisations to skip instead of using an env var
* update template id for sales
* Update link expiry time and headers
* Lower the threshold for testing
* Add issue explanation to the email
* Add how to fix
* update emails
* CLDC-2810 Create all addresses CSV (#1953)
* Add rake tasks for creating all addresses CSV
* Write headers if logs don't exist, update header names
* Rename method
* CLDC-2810 Correct addresses from csv (#1957)
* Updating importing lettings addresses form csv
* Add import_sales_addresses_from_csv rake
* Allow correcting addresses from both templates
* escape .
* Reinfer LA if the postcode hasn't changed
* Update labels and email content
* Update missing addresses threshold
* Remove unused env var
* Do not import logs if a duplicate log exists on the system
* deduplicate sales logs on import
* Only deduplicate logs created on new core
* Update test names
* create helper module to find all logs for a given user that have duplicates
write associated tests
update methods on user model to retrieve logs related to a given user
create a scope on logs to retrieve logs created by a given user
* check for duplicates and display notification banner on the index page where appropriate
tests to cover the desired behaviour
* alter duplicate logs helper to return duplicates in a format that can be passed through the params
update associated tests
* create user journey to view duplicate sets adn navigate to a given set
create a new route and add it to the link from the logs index page
create controller method and related view file
* update path for duplicate logs in routes, update factories, refactor duplicates scope to eliminate raw sql
* write tests for the duplicate logs index page
* correct linting errors
* minor amendments from code review.
* temp
* Refactor
* Create a scope for duplicate groups
* fixes
* lint
* Update controller specs
* Put duplicate sets count behind a feature flag
* refactor index
* Update duplicate_sets_count
* Update scopes
* tests
* pass in original log id
* tests
* Add duplicate index page content
---------
Co-authored-by: Kat <katrina@kosiak.co.uk>
* feat: wip commit
* feat: rest of assigned to filter behaviour
* feat: start to update tests
* feat: update tests
* feat: update tests
* refactor: linting
* feat: add new test
* feat: add new test
* feat: update filters_count
* feat: let support users see all users
* feat: let users select users from stock owners/managing agents too
* refactor: typo
* feat: update tests
* some minor refactoring
remove methods from child class that replicate methods on the parent class
tidy up check for nil
remove gubbins and inline method body given only used once
* update import services for lettings and sales to import creation method
write tests to cover this
* create sales log field import service and associated spec file, with methods and tests for importing the creation method of logs that have already been imported
* update lettings log field import service and related spec to allow importing creation method of logs
* use the methods dynamically created by active record in all relevant places, removing obsolete methods in teh process.
various tests tweaked to suppor this change.
rake task from another ticket folded into this ticket to prevent merge conflicts
* rename method for ruby conventions
* update PR for altered spec
upload id now decided to be a better indicator of bulk upload status, import service amended accordingly
tests updated in line with this
* update field import services in line with import services to use upload id rather than upload method as the source of truth for how a log was created
* slight refactor to reduce nesting and dodge linter complaints
* minor amendment to log creator spec in bulk upload to use enum dynamic methods
* Redirect lettings log to review page if the collection year is closed for editing
* Redirect sales log to review page if the collection year is closed for editing
* Update location default deactivation date
* Update location default deactivation date
* Do not display resources for a closed collection year
* lint
* Extract a method
* tests
* Update breadcrumbs for closed period
* form handler to return all questions from lettings forms for all years with ordering interleaved
functionality and tests
* refactor lettings log csv service and all associated tests
remove methods on log models when we can call them directly on associated models
update job to call the service directly with the collection of logs rather
minor modifications to the sales log csv service
update many test files to test the appropriate logic in the appropriate place
* tidying
final amendments to tests
remove commented code
rename variable
* change the position of the rent value check field in the headers
* CLDC-2492 add creation method field to logs (#1738)
* create migrations to add creation method fields to both log types
* add enum definition to logs for creation method
* upadte csv export services to retrieve creation method values direct from the log, remove methods previously used from the log model
* run migrations to update schema
* ensure that logs created via bulk upload have this set correctly when created
* Renme end_date to new_logs_end_date
* Display change buttons in CYA if the collection is still open for editing
* Allow navigating to question if the collection is still open for editing
* Allow logs to be edited if the date is before edit end date
* Update sales validation to allow editing existing logs
* update tests
* Update edit_end_date
* Update some test wording
* Update new logs end date
* tests
* protect against stack level errors in the case where a log has the status in progress but due to changes in the form should now have the status completed
further update the lettings log factory as the :completed trait was producing logs that were not copmlete
* add tests for calculate_status that implicitly also check if the factory traits for copmleted are working
also some minor renaming and refactoring
* correct linting errors
* remove status update but protect against error resurfacing in the future
* Remove bulk_upload_lettings_logs flag
* Add spacing above actions in data sharing agreement
* Add banner
* Add model validation
* fix log component
* Fix factory
* Put validation behind feature flag
* start adapting to data protection confirmation
* Make tests more generic
* Add tests for display_organisation_attributes
* Use data_protection_confirmed? method
* Address code review
* rubocop
* Prevent access to bulk upload pages when DPC not signed
* Address PO review
* feat: wip blank fields and dependent fields on upload tos ee if valid and can upload with missing info - this is not the exact ac on the ticket yet
* feat: update seed
* feat: wip commit
* feat: add postcodenk error so can clear on validation
* feat: add postcode validation back
* feat: move la vals to shared and move and add tests
* feat: add correct pluralisation to warning message
* feat: add blank compound invalid fields methods
* feat: update validations
* feat: update pluralisation
* refactor: lint
* feat: clear errors associated with blanked values so log status is set correctly on creation
* feat: validate instead
* feat: avoid duplicated errors
* feat: dont auto-refuse income, different to imports
* feat: validate after every blank method
* feat: delete la validator spec
* tests: update
* refactor: erblinting
* refactor: cleanup
* refactor: move pluralizer to helper
* feat: copy update
* feat: rename
* feat: refactor to avoid redundant re-validations and test
* refactor: rubocop
* test: update
* test: update
* test: update
* update sidekiq
* feat: clear errors
* feat: run clearing twice in case first clear creates different errors
* feat: remove moved file
* feat: undo validation file tweaks as shared/specific overlap could do with a more general refactor
* feat: update tests
* create a method on the FormHandler that returns the sales form questions for all years in the order that they appear in the form
* update csv email job to accomodate sales log export as well as lettings
add to tests to reflec the changes made
* write tests to cover the desired functionality of the SalesLogCsvService
* create the SalesLogCsvService
create a necessary method on the log to enable submission method to be included on the csv
derive values for the two halves of previous postcode for export
* add relevant links in the UI and pipe everything together in controllers
amend organisations controller to have flexibility to download logs of either type
add necessary methods to sales log controller, raising shared method to logs controller
update routing for amendments and additions
extract helper method to build urls for downloading logs within an organisation
* correct various linter complaints and tech review suggestions
* minor amendment to add old_id and reorder early columns
* undo my 'clever' refactor that broke things
* refactoring of csv service after some tech review and some UI testing in review app
* update tests to include a test of a full export and all values in teh csv
* correct minor routing error to ensure correct url is shown and tab selected after requesting csv email
* update organisations controller requests spec file to cover new functionality and make a minor amendment to authentication scope in the controller after error found in testing
* write request tests for the new functionality in the sales log controller, define authorisation in the controller
* minor correction after rubocop's kind suggestion'
* various corrections from first pass at PO, tech review, linter, etc
* refactor :ordered_sales_questions_for_all_years
* first pass at implementing flexible code-based form fixtures for testing
* second pass
* refactor all tests of :ordered_sales_questions_for_all_years to use new factories
* some refactoring in the testing of the csv service
* use that fact that params is always available in controllers and don't pass it around, inline some methods calls
* correct minor bug to ensure that "Return to logs" link returns to the correct index page
* remove reminder comments
* write further tests on the manipulation of questions into the csv headers, update factories of form constituents to allow the creation of forms with richer questions
* fix linter complaints
* minor alterations after rebase to account for changes made on other branches
* refactor after code review
* tweak fixtures after rebase containing alterations to the factory defaults
* write tests that invalid answers are cleared when the start date of a log is changed
* clear invalid answers when the start date of a log is changed
* correct error
* undo previous work to take new direction
* write tests to cover updated functionality of method in form to reset values when they are not routed to or when answers to radio questions are no longer valid
* update method in form to improve readability and to ensure that when the answers to radio questions are no longer valid, taht they are cleared
* add back an "end" deleted by mistake and fix a linting issue
* make some minor copy and variable name corrections
* fix a broken test after updates
* remove bulk upload tests for adding validations to unpermitted values to radio questions, these are now being cleared before validation
* fix some tests broken after rebasing
* add step to age questions in sales
* remove list of question ids from the form that should not be cleared, this information should be held on the questions themselves
* rename variables
* rename instance variable to be nice and long
* minor changes after rebase
* Set uprn_known to 1 when setting UPRN
* Move fake to dev+test gemfile block
So that we can use factories in local
* Move UPRN known and UPRN questions to single UPRN page
* Allow interruption screens to format money values
* Remove formatting from saleslog method
* Use money flag
* Use pre exhisting method to format values
* Update config/locales/en.yml
Co-authored-by: Arthur Campbell <51094020+arfacamble@users.noreply.github.com>
---------
Co-authored-by: Arthur Campbell <51094020+arfacamble@users.noreply.github.com>
* Format errors on money amounts with 2 decimals
* CLDC-1975 Format money input and error messages
* format discounted_ownership_value
* Format more locales
* Update lettings_log tests
* Fix import service spec
* add first page for bulk upload resume journey
* bulk upload resume handles upload again
* add confirm page to bulk upload resume journey
* replace placeholder count with correct value
* apply recommendation for bulk upload resume choice
* add how to fix bulk upload mailer
* integrate new bulk upload approve journey
* add missing bulk upload error mappings
* remove test
* prevent approve being called multiple times
* bulk upload creates invisible logs ahead of time
* work invisible logs into bulk upload flow
* sort errors so deterministic
* remove unused ensure
* remove expected_log_count and processed
- these fields are no longer used or needed
* introduce pending status
* swap visible for pending logs
* only show visible lettings logs
* hard code status filters
* remove unused model methods
* only show visible sales logs
* form controller ignores hidden logs
* locations and schemes only affect visible logs
* Add local authority links
* Display all local authorities for location
* set correct LA for log based on year
* Format local authorities for locations
* Rename variable
* Add import task for la links
* look at form start date because log start date might not be given
* Update app/models/lettings_log.rb
Co-authored-by: James Rose <james@jbpr.net>
* Update app/helpers/locations_helper.rb
Co-authored-by: James Rose <james@jbpr.net>
* Refactor
* Seed review app
* Change dates format
* Update the local authority link data
* Typo
* update mapping in Lettings logs
---------
Co-authored-by: James Rose <james@jbpr.net>
* feat: correctly infer las for 22/23 logs
* feat: copy changes to sales log
* feat: don't reset las when clash with postcode to make tests pass
* feat: align with similar PR for 23/24
* feat: tidying and add test
* refactor: lint
* refactor: use unless
* amend copy of hint text
* update test to reflect copy change
* display hint text on age question if that person is a child
* updates from code review
* update tests to reflect changes in class interfaces for page and question
* CLDC-853 Added hard validations for sales income2
* CLDC-853 Added soft validation for sales income2
* CLDC-853 Fix tests broken by new code
* CLDC-853 Add new tests for new page and refactor slightly
* CLDC-853 Fix linting errors
* CLDC-853 Rename migration and update schema version
* CLDC-853 Fix broken sales income2 test
* CLDC-853 Rename migration
* CLDC-853 Move income 2 to cya card 2 and commonise combined income validation
* CLDC-853 Actually use the validate_combined_income method
* combine duplicate methods after rebase, ensure hard validations are triggered on all relevant fields
* move validation on child income to financial validations to stop it being triggered on lettings logs, minor amendments to tests broken by changes
* revamp financial validations tests against income to reflect updates
* amend child income validation to reflect specifications and write tests to cover this validation
* correct linting errors and play a little code golf
* change copy for some validations, add sales log method and amend interruption screen helper to support this
* extract duplicate code to private method
* update buyer 1 and 2 income value check to be consistent
* remove ecstat from income checks, the only ecstat we care about is child which is dealt with elsewhere
* rename constant struct with same anme as existing variable
* amend tests to reflect the chagnes in validations and copy
* enable currency formatting of numbers for inserting into informative_text or title_text
* update evil test in form handler spec
* rebase and fix conflicts and tests
* change a variable name and correct minor rebase errors
* update interruption screen helper tests
* correct linting errors, minor test failure and typo
* add tests for new sales log method for formatting currency
* fix merge conflicts
---------
Co-authored-by: Arthur Campbell <arfa.camble@gmail.com>
* fix rebase conflicts
* fix rebase conflicts
* feat: update validation
* fix rebase conflicts
* fix rebase conflicts
* test: update
* fix rebase conflicts
* fix rebase conflicts
* fix rebase conflicts
* fix rebase conflicts
* fix rebase conflicts
* implement before validation check to ensure that if the user declares no mortgage will be used, mortgage value is set to 0
* fix rebase conflicts
* fix rebase conflicts
* remove comment
* fix rebase conflicts
* Update config/locales/en.yml
Co-authored-by: James Rose <james@jbpr.net>
* alter order of before validation methods on sales log so that derived fields are calculated after invalidated dependent fields are cleared, fix tests broken by this change
* add mortgage to derived variables so that mortgage value is set to 0 if a mortgage is not being used in the sale
* remove hard validation of range for discount question and allow shared validation of numeric questions to handle this case
* amend tests after rebasing
* fix tests after rebasing
* fix test and ensure schema correct after rebase
---------
Co-authored-by: natdeanlewissoftwire <nat.dean-lewis@softwire.com>
Co-authored-by: James Rose <james@jbpr.net>
* refactor with extract method
* can filter logs by bulk upload
* hide log creation button when viewing bulk upload
- this affects the logs index page filtering logs from a specific bulk
upload
* add info banner to bulk upload logs
* placeholder for bulk upload logs header
* when resuming bulk upload set filters
* fill place holder with remaining logs to fix
* add interstitial to resume if logs resolved
* after resolving bulk upload logs show interstitial
* fix linting error
* extract view variable to helper method
* Add min and max for age 1 validation
* Add retirement value check column to the sales table
* Add retirement value check question and page
* Add retirement value check pages to household characteristics
* Reuse soft validations for retirement in sales
* Update person_indexes
* Add old_persons_shared_ownership validation
* Display the retirement value check in correct places depending on the joinpur value
* Update page IDs
* It's always joint purchase if age 2 is displayed
* lint
* lint
* add class to create logs from bulk upload
* create logs when processing bulk uploads
* remove bulk_upload_id from csv output
* create bulk upload logs only if all valid
- this will be changed later to allow for partial logs
- and only to create logs when a threshold has been met
* add method to blank invalid non setup fields
* bulk upload log creation blanks invalid fields
* fix incorrect logic for bulk upload renewal
* fix linting
* bulk upload log creation fail logs to sentry
* fix bulk upload line ending parsing
* extract bulk uploading csv parsing to class
* use csv parser in log creator
* change handle line endings mechanism
- we now strip all windows line endings for unix based line endings
- this normalises things making it simpler
Rachael Booth
kosiakkatrina
Jack
Arthur Campbell
natdeanlewissoftwire
Phil Lee
David May-Miller