submit-social-housing-lettings-and-sales-data

Commit Graph

Author	SHA1	Message	Date
Arthur Campbell	8c34719987	CLDC-2291 refactor and minor updates to lettings log csv service (#1719 ) * form handler to return all questions from lettings forms for all years with ordering interleaved functionality and tests * refactor lettings log csv service and all associated tests remove methods on log models when we can call them directly on associated models update job to call the service directly with the collection of logs rather minor modifications to the sales log csv service update many test files to test the appropriate logic in the appropriate place * tidying final amendments to tests remove commented code rename variable * change the position of the rent value check field in the headers * CLDC-2492 add creation method field to logs (#1738) * create migrations to add creation method fields to both log types * add enum definition to logs for creation method * upadte csv export services to retrieve creation method values direct from the log, remove methods previously used from the log model * run migrations to update schema * ensure that logs created via bulk upload have this set correctly when created	2 years ago
natdeanlewissoftwire	b0271ea0ec	CLDC-2257 Refactor filters (#1675 ) * feat: split lettings and sales filters * feat: remove logs filter * refactor: type -> filter_type * refactor: slight cleanup * feat: update csv download * feat: set org filters to mirror primary sales and lettings * refactor: linting * feat: fix filter types in org controller - can be changed to org specific labels in future if needed rather than copying filters from primary sales/lettings tabs * refactor: initalize filter_type * refactor: lint * refactor: combine filter_manager into filter_service * refactor: rename to filter manager * refactor: DRY session name filter logic * feat: share session filter name logic * feat: further refactor * refactor: simplify * feat: revert change to filter manager	2 years ago
Arthur Campbell	40cf0c10c2	CLDC-1633 build feature csv download of sales logs (#1568 ) * create a method on the FormHandler that returns the sales form questions for all years in the order that they appear in the form * update csv email job to accomodate sales log export as well as lettings add to tests to reflec the changes made * write tests to cover the desired functionality of the SalesLogCsvService * create the SalesLogCsvService create a necessary method on the log to enable submission method to be included on the csv derive values for the two halves of previous postcode for export * add relevant links in the UI and pipe everything together in controllers amend organisations controller to have flexibility to download logs of either type add necessary methods to sales log controller, raising shared method to logs controller update routing for amendments and additions extract helper method to build urls for downloading logs within an organisation * correct various linter complaints and tech review suggestions * minor amendment to add old_id and reorder early columns * undo my 'clever' refactor that broke things * refactoring of csv service after some tech review and some UI testing in review app * update tests to include a test of a full export and all values in teh csv * correct minor routing error to ensure correct url is shown and tab selected after requesting csv email * update organisations controller requests spec file to cover new functionality and make a minor amendment to authentication scope in the controller after error found in testing * write request tests for the new functionality in the sales log controller, define authorisation in the controller * minor correction after rubocop's kind suggestion' * various corrections from first pass at PO, tech review, linter, etc * refactor :ordered_sales_questions_for_all_years * first pass at implementing flexible code-based form fixtures for testing * second pass * refactor all tests of :ordered_sales_questions_for_all_years to use new factories * some refactoring in the testing of the csv service * use that fact that params is always available in controllers and don't pass it around, inline some methods calls * correct minor bug to ensure that "Return to logs" link returns to the correct index page * remove reminder comments * write further tests on the manipulation of questions into the csv headers, update factories of form constituents to allow the creation of forms with richer questions * fix linter complaints * minor alterations after rebase to account for changes made on other branches * refactor after code review * tweak fixtures after rebase containing alterations to the factory defaults	2 years ago
James Rose	6564ca3587	CDS: Update export functionality to allow for multiple collection years at once (#1491 ) * Remove CSV export * Add collection to lettings export * Use staging bucket	2 years ago
Phil Lee	454df8389e	CLDC-2135 Bulk upload resume with how fix (#1503 ) * add first page for bulk upload resume journey * bulk upload resume handles upload again * add confirm page to bulk upload resume journey * replace placeholder count with correct value * apply recommendation for bulk upload resume choice * add how to fix bulk upload mailer * integrate new bulk upload approve journey * add missing bulk upload error mappings * remove test * prevent approve being called multiple times * bulk upload creates invisible logs ahead of time * work invisible logs into bulk upload flow * sort errors so deterministic * remove unused ensure * remove expected_log_count and processed - these fields are no longer used or needed * introduce pending status * swap visible for pending logs * only show visible lettings logs * hard code status filters * remove unused model methods * only show visible sales logs * form controller ignores hidden logs * locations and schemes only affect visible logs	2 years ago
Arthur Campbell	a2b684b123	CLDC-1826 lettings log codes only download (#1268 ) * update seeds to add self in review env, change spec to reflect this, update config yml to allow csv exports in review * update interface of relevant methods EmailCsvJob, LettingsLog.to_csv and LettingsLogCsvService consume codes_only flag * update tests including adding a new csv file to test against * update LettingsLogCsvService to output codes only csv * correct minor error and linting * enable codes only download in UI - add link on lettings log index page - pass codes_only flag through params in relevant links and methods - convert flag to boolean in controller methods * ensure link displayed successfully for all renderings of logs_list and params passed through relevant methods in organisations controller * fix existing tests * correct linting thing * correct linting error * update tests for lettings log controller * correct linting errors * update organisations controller tests * make minor changes after code review * remove changes made for testing on review app * make codes only download visible to support users only * change variable names throughout after info on rauby/rails naming conventions, update tests for change in who can view codes only download link * rework csv service for readability, remove delegating methods from lettings log to keep all code to do with mapping between our domain and desired export format in one place * update test name * correct a small typo and remove a duplicated method after clever git merge conflict suggestion * point review app at staging csv bucket for csv download * change variables named codes_only_export to codes_only to avoid inconsistency * write tests to ensure that differetn user roles have the correct permissions around csv download * ensure that non support users may not download codes only exports * correct a small error in a previous commit * correct minor linting error	2 years ago
James Rose	68faee2efd	Schedule data export tasks in sidekiq-cron (#952 ) * Schedule data export tasks in sidekiq-cron - Split combined XML and CSV data jobs into seperate background jobs - Add sidekiq-cron to the project - Schedule new XML and CSV jobs to run every day at 5am - Adjust data export Rake tasks to use new job * Add specs for DataExport{Xml,Csv}Job * x * Run bundle install * Change review app export bucket name --------- Co-authored-by: Kat <katrina@kosiak.co.uk>	2 years ago
Phil Lee	197e2ae4cb	[1778] Bulk upload errors (#1091 ) * lockdown bulk upload routes * able to view lettings bulk upload errors * add error count to bulk upload results * coverage for bulk upload filename on results * group bulk upload errors by row on results * able to view bulk upload sales results * scope lettings and sales bulk upload results * fix linting * call service correctly in test * add bulk upload sales questions mapping * appease linter * bulk upload error shows correct question - depending on log type it will show relevant question for the field concerned * use local disk for bulk upload for dev env - this saves the need to connect to S3 to play with bulk upload in dev environment * improve namespacing of classes * add job to process bulk uploads * use local disk storage for dev file upload * fix test * use inline active job queue_adapter for dev * use test active job queue adapter for test env * remove rubocop violation * delete bulk upload from disk after processing * populate errors with cell, row + metadata * update error message with something meaningful * shim in sales validator * able to parse sales bulk uploads * change migration to add purchase_code to errors * bulk upload error component renders purchaser code - when a sales log * populate purchaser_code for bulk upload errors - when log type is sales * remove superfluous private method	2 years ago
James Rose	d3075e4e43	Revert to `f4e6d6d6bb` (#887 )	2 years ago
natdeanlewissoftwire	f162f880ae	Revert "Merge remote-tracking branch 'origin/main' into CLDC-1429-output-la-code" This reverts commit `626e2bd1a6`, reversing changes made to `f86e13cea7`.	2 years ago
James Rose	7927cf99da	Use service credentials to upload and presign CSV exports to S3 (#868 )	2 years ago
James Rose	947b2287f6	Use correct mail method to deliver CSV email	2 years ago
Dushan	67ce147fc9	Extract base log and create sales log (#858 ) * Add abstract log class and sales log class Created a parent log class for sales log and lettings log. Any bits common to both sales and lettings can live in the parent class. As the sales log functionality is built up any commonalities with lettings log can be extracted into the parent log class. The sales log model is set up without a json form and instead the form is defined in code - like the setup section of the lettings log. * update sales logs controller * update lettings controller specs * update filter method name * update organisations controller * use lettings method * Add deleted tests back * lint Co-authored-by: Kat <katrina@madetech.com> Co-authored-by: Kat <kosiak.katrina@gmail.com>	2 years ago
Sam	a92fb1030c	CLDC-1362 Improve CSV download performance (#851 ) * Replaced log CSV direct download with email * Tidy up authorization of organisations controller - We already have a method to authenticate the scope of the user, so we can reuse that. * Use Rails routes instead of absolute paths for CSV download links * Introduce base NotifyMailer to to abstract away shared Notify mail functionality * Fix mailer spec name * Add worker instance to PaaS manifest * Add CSV download bucket instance name into environment * Update tests for improved search term handling * Fix download mailer tests * Clarifying comments Co-authored-by: natdeanlewissoftwire <nat.dean-lewis@softwire.com> Co-authored-by: James Rose <james@jbpr.net>	2 years ago
baarkerlounger	4897e2cd76	Add application skeleton to repo	3 years ago

15 Commits (e7684fc2ddb20efcfc28228480420049f03c3169)