* Refactor manifest creation into a separate job
* Add users export service
* Call user export service
* Rename exports table
* Update data export job spec
* Update naming
* Refactor shared logic into parent class
* Update initialize
* Allow exporting users individually
* Update data export task tests
* Move method and update task argument
* Add phone extension to the user export
* Add static period to filename
* Make recent logs export depend on the collection
* CLDC-3534 Export organisation data (#2599)
* Add organisation export service
* Call organisations export and write manifest
* Add some additional fields to export
* Add period to organisation export filename
* Update provider_type and add new fields
* Filter exports by the collection
* Update tests
* Update fields exported in lettings export (#2652)
* Add new fields for user ids (#2661)
* Undo lettings export field changes
* CLDC-2093 Merge organisations page - view all merge requests (#2561)
* CLDC-3585 Update absorbing organisation question (#2564)
* Update user permissions and absorbing org question order
* Update absorbing organisation question and routing
* CLDC 2094: View a merge request (#2565)
* CLDC-3584 Update merging orgs question (#2567)
* Update merging organisations question
* Update rebase tests
* Update routing between CYA and questions
* CLDC-3586 Update merge date merge request question (#2571)
* Remove unused paths and update merge date
* lint
* Create delete merge request functionality (#2568)
* CLDC-3588: Add helpdesk ticket question (#2572)
* Merge request fixes (#2578)
* Add cancel button to merging orgs and update merging orgs selection
* Update submit button text
* Update back buttons
* CLDC-2101 Add begin merge (#2575)
* Calculate merge request status
* Add start merge and merge request job
* Update merge request when it gets processed
* Refactor and display a banner for failed requests
* update test
* Update change links
* Update copy for error message (#2583)
* Do not display cancel button if there's no submit (#2584)
* Update hint date (#2588)
* CLDC-2100++ Add success notification after delete (#2589)
* Update helpdesk_ticket.html.erb
* CLDC-2094++ Merge details page adjustments (#2593)
* Change page name and hide buttons during processing status
* Update test
* Make merge request status dynamic (#2592)
* Make merge request status dynamic
* Update tests
* Keep check answers referrer when validation is hit (#2594)
* Keep check answers referrer when validation is hit
* Remove absorbing org from merging organisations list
* Update text to handle singular merge requests
* Add merge start confirmation page (#2601)
* Set merge request as non processing if it fails (#2598)
* Set merge request to no longer processing if it fails
* clear last_failed_attempt as soon as we start processing the merge
* CLDC-3603 View merged users outcomes (#2602)
* Update user outcomes view link
* Add user outcomes page
* Set user outcomes before merge and on merge fail
* Update hardcoded total user count
* Account for singular user numbers
* CLDC-3603 View scheme outcomes (#2604)
* Update scheme outcomes view link
* Add scheme outcomes page
* Update scheme outcome before merge and on merge fail
* Update back links in merge outcomes pages (#2611)
* CLDC-3605 View relationships page (#2606)
* CLDC-3609 Add existing absorbing organisation merge request question (#2600)
* Add existing absorbing organisation page and field
* Update status calculation and add new question to check answers
* Call correct merge service flow
* Update test
* Update test
* Return correct status when existing_absorbing_organisation is false (#2617)
* Change styling of relationship outcomes page (#2619)
* CLDC-3604 View logs outcomes (#2618)
* Update view logs outcomes link
* Add logs outcomes page
* Set logs outcome before merge and on merge fail
* Update logs outcomes text
* Update line break
* Fix incorrect filtering of relationships and count
* Fix test
* Fix test
* Fix test
* Make example date depend on collection year (#2620)
---------
Co-authored-by: Manny Dinssa <44172848+Dinssa@users.noreply.github.com>
* Filter out empty options from selected filters
* Redirect to year filter if it is not selected
* Add remaining filter questions
* Display lettings filters CYA
* Add cancel, conditionally display send email
* Refactor
* Add removed tests
* Update method name
* Add sales logs filters
* Update tests
* Set label size on filter questions
* Refactor
* Deduplicate filter views
* Rename methods
* Authenticate pages
* Update filtering per organisation
* Update routing from CYA
* lint
* Refactor check_your_answers_filters_list
* Redirect if year is not given for sales csv
* Update formatted organisations methods
* Update missing and wrong copy
* Update cancel and back buttons
* CLDC-3442 Download logs csv per year (#2404)
* Specify year in lettings csv download service
* Specify year in sales csv download service
* Define non_question_fields per year
* Tidy up missing fields and tests
* Fix request tests
* Refactor
* Add missing param
* CLDC-3348 Update sales CSV for support (#2407)
* Update sales csv for support
* CLDC-3352 Update sales CSV address order (#2410)
* Update sales support csv address ordering
* Refactor
* CLDC-3367 Rename sales support CSV fields (#2413)
* Rename sales support CSV fields
* Update sales CSV user fields order (#2416)
* CLDC-2586 Rename scheme sensitive (#2405)
* Rename scheme sensitive in lettings csv
* Rename scheme sensitive in schemes csv
* CLDC-3347 Show soft validations in support CSVs only (#2406)
* Show soft validations in support csvs only
* CLDC-3352 Update lettings CSV address order (#2409)
* Update lettings support csv address ordering
* Refactor
* CLDC-3415/3421/3422 Add lettings CSV updates (#2415)
* Add first_time_property_let_as_social_housing to coordinators csv
* Update lettings CSV user fields order
* Remove new_old from lettings CSV
* Remove duplicate rent value check from CSV (#2423)
* Refactor CSV service (#2425)
* Extract and move lettings methods
* Extract and more sales methods
* feat: add schemes and locations download links and pages
* feat: update current path helper
* feat: update tests for different user visibility levels
* feat: update search caption tests
* refactor: lint tests
* refactor: lint tests
* git: revert unintentional inclusion
* feat: update tests
* refactor: lint
* feat: DRY up routing
* refactor: lint
* feat: add csv confirmation view
* feat: add scheme csv service
* feat: rename
* feat: update csv service
* feat: update csv service
* feat: update controller and rename view
* feat: update view
* refactor: lint
* feat: show correct headers in csv
* feat: add locations and combined csv behaviour
* feat: remove redundant user instance variable
* feat: add scheme csv service spec
* feat: add scheme email csv job tests
* feat: update filters in spec
* refactor: move scheme_email_csv_job_spec.rb
* feat: update spec
* refactor: remove blank line
* feat: add nowrap to all download links
* feat: update org schemes controller with org schemes (and rename for clarity)
* feat: update link indentation and spec
* feat: only include location LA name, and rename to location_local_authority
* feat: update seed locations with westminster local authorities to avoid similar confusion to some that arose in PO review
* feat: display multiple active periods on a single line
* feat: display multiple active periods on a single line
* feat: update line spacing in search captions
* feat: replace 2/3 with full column in download page
* feat: move scheme alphabeticising into manager
* feat: update tests now search/filterless copy has changed
* refactor: lint
* refactor: lint
* refactor: lint
* feat: add filter alphabeticising test
* feat: correct spacing
* Add missing addresses csv job
* Update missing addresses csv service methods
* Add rake task
* Update the job to send missing town or city templates
* Update service to create mising town or city templates
* Add send missing town or city csv rake task
* Add IDs to the CSVs
* Put all log in the same csv
* Add issue type column
* Write wrong uprn logs to csv
* Add mailer methods
* Skip uprn issue for specified orgs
* Add sales csv rake task
* set SKIP_UPRN_ISSUE_ORG_IDS on review apps
* test
* Update notify template IDs for testing
* Initialize service with organisation instead of a hash
* Add expiration time to url
* Add optional tags and remove LA from csv
* Extract log to csv methods
* Update casing
* Update old IDs in factories
* Move constant
* Extract some repeating scopes
* Pass in organisations to skip instead of using an env var
* update template id for sales
* Update link expiry time and headers
* Lower the threshold for testing
* Add issue explanation to the email
* Add how to fix
* update emails
* CLDC-2810 Create all addresses CSV (#1953)
* Add rake tasks for creating all addresses CSV
* Write headers if logs don't exist, update header names
* Rename method
* CLDC-2810 Correct addresses from csv (#1957)
* Updating importing lettings addresses form csv
* Add import_sales_addresses_from_csv rake
* Allow correcting addresses from both templates
* escape .
* Reinfer LA if the postcode hasn't changed
* Update labels and email content
* Update missing addresses threshold
* Remove unused env var
* feat: update application to use non paas and paas configuration services, add helper to determine if platform is paas
feat: don't double-instantiate the configuration service
feat: add self. to is_paas? definition, and require module in rack_attack
chore: lint
* test: update suite to work with new paas/env config service logic
test: new PlatformHelper
* fix: update full import tests to check paas and non-paas config services
* fix: update test descriptions and remove unrequired line
---------
Co-authored-by: Sam Seed <sam.seed@softwire.com>
* form handler to return all questions from lettings forms for all years with ordering interleaved
functionality and tests
* refactor lettings log csv service and all associated tests
remove methods on log models when we can call them directly on associated models
update job to call the service directly with the collection of logs rather
minor modifications to the sales log csv service
update many test files to test the appropriate logic in the appropriate place
* tidying
final amendments to tests
remove commented code
rename variable
* change the position of the rent value check field in the headers
* CLDC-2492 add creation method field to logs (#1738)
* create migrations to add creation method fields to both log types
* add enum definition to logs for creation method
* upadte csv export services to retrieve creation method values direct from the log, remove methods previously used from the log model
* run migrations to update schema
* ensure that logs created via bulk upload have this set correctly when created
* create a method on the FormHandler that returns the sales form questions for all years in the order that they appear in the form
* update csv email job to accomodate sales log export as well as lettings
add to tests to reflec the changes made
* write tests to cover the desired functionality of the SalesLogCsvService
* create the SalesLogCsvService
create a necessary method on the log to enable submission method to be included on the csv
derive values for the two halves of previous postcode for export
* add relevant links in the UI and pipe everything together in controllers
amend organisations controller to have flexibility to download logs of either type
add necessary methods to sales log controller, raising shared method to logs controller
update routing for amendments and additions
extract helper method to build urls for downloading logs within an organisation
* correct various linter complaints and tech review suggestions
* minor amendment to add old_id and reorder early columns
* undo my 'clever' refactor that broke things
* refactoring of csv service after some tech review and some UI testing in review app
* update tests to include a test of a full export and all values in teh csv
* correct minor routing error to ensure correct url is shown and tab selected after requesting csv email
* update organisations controller requests spec file to cover new functionality and make a minor amendment to authentication scope in the controller after error found in testing
* write request tests for the new functionality in the sales log controller, define authorisation in the controller
* minor correction after rubocop's kind suggestion'
* various corrections from first pass at PO, tech review, linter, etc
* refactor :ordered_sales_questions_for_all_years
* first pass at implementing flexible code-based form fixtures for testing
* second pass
* refactor all tests of :ordered_sales_questions_for_all_years to use new factories
* some refactoring in the testing of the csv service
* use that fact that params is always available in controllers and don't pass it around, inline some methods calls
* correct minor bug to ensure that "Return to logs" link returns to the correct index page
* remove reminder comments
* write further tests on the manipulation of questions into the csv headers, update factories of form constituents to allow the creation of forms with richer questions
* fix linter complaints
* minor alterations after rebase to account for changes made on other branches
* refactor after code review
* tweak fixtures after rebase containing alterations to the factory defaults
* add first page for bulk upload resume journey
* bulk upload resume handles upload again
* add confirm page to bulk upload resume journey
* replace placeholder count with correct value
* apply recommendation for bulk upload resume choice
* add how to fix bulk upload mailer
* integrate new bulk upload approve journey
* add missing bulk upload error mappings
* remove test
* prevent approve being called multiple times
* bulk upload creates invisible logs ahead of time
* work invisible logs into bulk upload flow
* sort errors so deterministic
* remove unused ensure
* remove expected_log_count and processed
- these fields are no longer used or needed
* introduce pending status
* swap visible for pending logs
* only show visible lettings logs
* hard code status filters
* remove unused model methods
* only show visible sales logs
* form controller ignores hidden logs
* locations and schemes only affect visible logs
* update seeds to add self in review env, change spec to reflect this, update config yml to allow csv exports in review
* update interface of relevant methods
EmailCsvJob, LettingsLog.to_csv and LettingsLogCsvService consume codes_only flag
* update tests including adding a new csv file to test against
* update LettingsLogCsvService to output codes only csv
* correct minor error and linting
* enable codes only download in UI
- add link on lettings log index page
- pass codes_only flag through params in relevant links and methods
- convert flag to boolean in controller methods
* ensure link displayed successfully for all renderings of logs_list and params passed through relevant methods in organisations controller
* fix existing tests
* correct linting thing
* correct linting error
* update tests for lettings log controller
* correct linting errors
* update organisations controller tests
* make minor changes after code review
* remove changes made for testing on review app
* make codes only download visible to support users only
* change variable names throughout after info on rauby/rails naming conventions, update tests for change in who can view codes only download link
* rework csv service for readability, remove delegating methods from lettings log to keep all code to do with mapping between our domain and desired export format in one place
* update test name
* correct a small typo and remove a duplicated method after clever git merge conflict suggestion
* point review app at staging csv bucket for csv download
* change variables named codes_only_export to codes_only to avoid inconsistency
* write tests to ensure that differetn user roles have the correct permissions around csv download
* ensure that non support users may not download codes only exports
* correct a small error in a previous commit
* correct minor linting error
* Schedule data export tasks in sidekiq-cron
- Split combined XML and CSV data jobs into seperate background jobs
- Add sidekiq-cron to the project
- Schedule new XML and CSV jobs to run every day at 5am
- Adjust data export Rake tasks to use new job
* Add specs for DataExport{Xml,Csv}Job
* x
* Run bundle install
* Change review app export bucket name
---------
Co-authored-by: Kat <katrina@kosiak.co.uk>
* lockdown bulk upload routes
* able to view lettings bulk upload errors
* add error count to bulk upload results
* coverage for bulk upload filename on results
* group bulk upload errors by row on results
* able to view bulk upload sales results
* scope lettings and sales bulk upload results
* fix linting
* call service correctly in test
* add bulk upload sales questions mapping
* appease linter
* bulk upload error shows correct question
- depending on log type it will show relevant question for the field
concerned
* use local disk for bulk upload for dev env
- this saves the need to connect to S3 to play with bulk upload in dev
environment
* improve namespacing of classes
* add job to process bulk uploads
* use local disk storage for dev file upload
* fix test
* use inline active job queue_adapter for dev
* use test active job queue adapter for test env
* remove rubocop violation
* delete bulk upload from disk after processing
* populate errors with cell, row + metadata
* update error message with something meaningful
* shim in sales validator
* able to parse sales bulk uploads
* change migration to add purchase_code to errors
* bulk upload error component renders purchaser code
- when a sales log
* populate purchaser_code for bulk upload errors
- when log type is sales
* remove superfluous private method
* Add abstract log class and sales log class
Created a parent log class for sales log and lettings log. Any bits common
to both sales and lettings can live in the parent class. As the sales log
functionality is built up any commonalities with lettings log can be extracted
into the parent log class. The sales log model is set up without a json form
and instead the form is defined in code - like the setup section of the lettings
log.
* update sales logs controller
* update lettings controller specs
* update filter method name
* update organisations controller
* use lettings method
* Add deleted tests back
* lint
Co-authored-by: Kat <katrina@madetech.com>
Co-authored-by: Kat <kosiak.katrina@gmail.com>
* Replaced log CSV direct download with email
* Tidy up authorization of organisations controller
- We already have a method to authenticate the scope of the user, so we can reuse that.
* Use Rails routes instead of absolute paths for CSV download links
* Introduce base NotifyMailer to to abstract away shared Notify mail functionality
* Fix mailer spec name
* Add worker instance to PaaS manifest
* Add CSV download bucket instance name into environment
* Update tests for improved search term handling
* Fix download mailer tests
* Clarifying comments
Co-authored-by: natdeanlewissoftwire <nat.dean-lewis@softwire.com>
Co-authored-by: James Rose <james@jbpr.net>
Kat
natdeanlewissoftwire
Chirag-Bhatti
Arthur Campbell
James Rose
Phil Lee
natdeanlewissoftwire
Dushan
Sam
baarkerlounger