Index to increase the performance

This commit is contained in:
2026-07-21 22:07:45 +05:30
parent f0481d090c
commit 4cbd510b85
3 changed files with 576 additions and 0 deletions

View File

@@ -0,0 +1,132 @@
# Database performance audit
## Scope and safety
This audit covers the PostgreSQL schema, indexes, constraints, views, functions, and their relationship to `src/matrix/nimble/conf/nimble.qry`, with particular focus on Visit MIS and Dashboard.
The catalog inspection is read-only. No indexes, views, functions, or PostgreSQL settings have been changed. The audited local database is only about 30 MB and its core business tables currently contain almost no live data, so local execution times cannot represent production. Final index selection must be validated with production statistics and `EXPLAIN (ANALYZE, BUFFERS)` captured for representative parameters.
## Inventory
- PostgreSQL: 18.4
- User tables: 120
- Normal views: 80
- Materialized views: 2
- User functions: 220
- Indexes: 187
- Function statistics: disabled (`track_functions = none`)
- Statement statistics: unavailable (`pg_stat_statements` is not preloaded)
The raw catalog exports are under `target/database-audit/` and can be regenerated with `matrix.nimble.DatabaseAuditRunner`.
The subsequent analysis of the complete `matrix.sql` schema dump confirmed that `portfolio_bank_branch` and `colony_allocation` are normal views rather than tables. Index recommendations therefore target their underlying tables (`portfolio_branch`, `bank_branch`, `portfolio`, `colony`, `belt_allocation`, and `verifier`).
Generated index script: `docs/production-indexes.sql`. It contains preflight
checks, high-confidence Visit MIS/Dashboard indexes, foreign-key support indexes,
statistics refresh, postflight validation, and a commented rollback reference.
## Application-to-database mapping
| Feature | Application query | Database entry point |
|---|---|---|
| Visit MIS | Query306 | `monthvisitmis(...)` |
| Dashboard initial load | Query473 | `dashboard()` |
| Dashboard partial refresh | Query525 | `dashboardpartial(...)` |
`nimble.qry` contains about 460 application query definitions. Its most frequently referenced core relation is `main`, followed by document/bank-report relations, `app_user`, `main_mainopr`, `verifier`, `temp_reporting`, and `portfolio_bank_branch`. This is source frequency, not runtime frequency; production statement statistics are required for workload ranking.
## Visit MIS findings
### Execution shape
`monthvisitmis(...)` loops over each requested day and performs an aggregate against `visit_mis`. For every verifier/operator row returned, it runs a second query to find `max(doneon)`. This is an N+1 query pattern: cost grows with both date range and verifier count.
`vervisitmis(...)` has the same per-verifier `max(doneon)` lookup and an additional per-verifier aggregate against `verifier_monthly_data`. It also dynamically switches between `current_visit_mis` and `visit_mis`.
Both functions build complete HTML reports inside PL/pgSQL through repeated text concatenation. This increases database CPU and memory usage and prevents the Java/UI layer from receiving structured rows. It is not the first optimization target, but should ultimately be replaced with a set-returning/data function and JSP rendering.
### View expansion
`visit_mis` and `current_visit_mis` each expand into three `UNION ALL` branches for residence, office, and property visits. Every branch joins:
- `main_mainopr`, itself a join of `main` and `main_operations` by `uuid`
- `portfolio_bank_branch` by `portbranch_id`
- `verifier` by the visit-specific verifier column
- `app_user` by verifier operator
- `riskalerts` by `(uuid, visit)`
Every branch also performs a correlated zone aggregation against `colony_allocation` by `verifier_id`.
`verifier_monthly_data` is another three-branch `UNION ALL` over `main`, filtered by `receivedate`, visit flag, verifier, and same-address flag.
### Index gaps and mismatches
These are candidates to validate on production, not DDL to run blindly:
1. `riskalerts` has separate indexes on `uuid` and `visit`, although the views join using both. A composite index beginning with `(uuid, visit)` matches the access path better.
2. `colony_allocation` is a view. Its base table `belt_allocation` has an index on `verifier_id`, but lacks an index beginning with `belt_id` for its join to `colony`; the generated script adds `(belt_id, verifier_id)`.
3. `main_operations` has single-column indexes on `oprsendon`, `sdoneon`, `sdone`, and `fileclosed`, but the report predicates combine time/status data after joining by `uuid`. Production plans are needed to decide between composite and partial indexes.
4. `main` has individual verifier indexes and a `receivedate` index. `verifier_monthly_data` combines each verifier with `receivedate`, a visit flag, and same-address status, so one-column indexes may cause excessive filtering after index access.
5. `portfolio_bank_branch` is a view. Its base table `portfolio_branch` already has a primary key on `portbranch_id`; additional indexes are recommended for its portfolio/active and bank-branch access paths.
6. Several existing indexes duplicate or overlap: `main.bank_branch_id` has two btree indexes, and `main_operations.uuid` has both a primary btree index and a hash index. Production usage and index size/write cost should be checked before removing anything.
### Primary query rewrite
The per-verifier `max(doneon)` must be folded into the main grouped query as `max(doneon) FILTER (...)`. Monthly visit count/limit should also be pre-aggregated once and joined by verifier. This changes roughly `1 + N + N` queries per report/day into one grouped query per day, or preferably one query for the entire requested range.
The date loop can be replaced by a single `generate_series` relation joined to the source data, or by grouping source rows by reporting date. The best variant depends on the required backlog semantics and will be verified against current output before replacement.
## Dashboard findings
`dashboard()` executes at least ten child functions serially:
1. `dashvisits()`
2. `dashscannerhourly()`
3. `dashoperatorhourly('opr')`
4. `dashrihhourly()`
5. `dashoperatorhourly('edp')`
6. `dashhourlytobesolved()`
7. `dashriskandphoto()`
8. `dashageing()`
9. `dash3monthsfigures(1, 0)`
10. `dash3monthsfigures(0, 0)`
The major repeated work is:
- `dashvisits()` repeatedly counts the same expanded `current_visit_mis` and `scanning_grid` views for different categories.
- `dashageing()` scans `scanning_grid` about seven times to produce age buckets.
- `dashboardpartial()` calls `dash3monthsfigures()` twice for one widget refresh.
- Dashboard views expand through `main`, `main_operations`, `temp_reporting`, `riskalerts`, user/company/branch/portfolio lookups, and report tables.
These should be consolidated using conditional aggregates (`count(*) FILTER (WHERE ...)`) so each source is scanned once per widget. The initial dashboard response should then call fewer functions or one data-returning function. UI polling should avoid starting a refresh while the previous request is still running.
## Schema and observability findings
- The database relies heavily on views and PL/pgSQL functions, including dynamic SQL. PostgreSQL dependency metadata cannot discover relations embedded only in dynamic SQL, so function definitions must be reviewed in addition to catalog dependencies.
- A number of foreign keys lack a leading index. These matter mainly for parent updates/deletes and child-side joins; each must be ranked by actual workload rather than indexed automatically.
- Core tables contain some redundant indexes, while some logical join keys have no visible supporting index.
- `track_functions = none` prevents attribution of time to the 220 functions.
- `pg_stat_statements` is unavailable, preventing reliable ranking by total time, mean time, calls, rows, and temporary I/O.
## Required production evidence
Before applying point 5, collect during a normal busy period:
1. Row counts and relation/index sizes for the Visit MIS and Dashboard base tables.
2. `pg_stat_user_tables` and `pg_stat_user_indexes` snapshots before and after the period.
3. `EXPLAIN (ANALYZE, BUFFERS, WAL, SETTINGS, VERBOSE)` for Visit MIS source aggregates using representative branch, verifier, and date-range values.
4. Slow statement data from `pg_stat_statements` after enabling it through the normal PostgreSQL change process.
5. Function timing after setting `track_functions = pl` (or `all`) through the normal production change process.
6. PostgreSQL logs with `log_min_duration_statement` set to a suitable temporary threshold, if extensions/settings cannot be enabled immediately.
Do not run `EXPLAIN ANALYZE` on production-changing functions. Extract and explain their underlying `SELECT` statements inside a read-only transaction.
## Recommended implementation order for point 5
1. Enable observability and capture a baseline.
2. Add only production-plan-confirmed indexes, using `CREATE INDEX CONCURRENTLY` and checking duplicate/overlapping indexes first.
3. Rewrite Visit MIS to eliminate N+1 lookups and repeated date scans while preserving report totals exactly.
4. Consolidate Dashboard counts into single-pass aggregates.
5. Return structured data from new versioned functions and render HTML in the application; retain old functions during comparison.
6. Run old and new implementations side by side for representative dates/branches and compare every total before switching.
7. Load-test with at least 20 concurrent sessions and monitor database CPU, pool wait time, query latency, temporary I/O, and heap usage.

245
docs/production-indexes.sql Normal file
View File

@@ -0,0 +1,245 @@
-- Cygnus PostgreSQL index migration
-- Derived from matrix.sql, nimble.qry, Visit MIS, and Dashboard functions/views.
--
-- IMPORTANT
-- 1. Run each CREATE INDEX as a separate statement. Do not wrap this file in BEGIN/COMMIT.
-- 2. CREATE INDEX CONCURRENTLY minimizes blocking but still consumes CPU, I/O and disk.
-- 3. Run during a lower-traffic window and monitor replication lag and free disk space.
-- 4. IF NOT EXISTS checks the name only. Run the preflight query first to inspect equivalent indexes.
-- 5. These statements do not change table columns, constraints, or stored data.
-- ---------------------------------------------------------------------------
-- Preflight: current sizes and existing definitions
-- ---------------------------------------------------------------------------
SELECT
n.nspname AS schema_name,
t.relname AS table_name,
i.relname AS index_name,
pg_size_pretty(pg_relation_size(i.oid)) AS index_size,
pg_get_indexdef(i.oid) AS definition
FROM pg_index x
JOIN pg_class t ON t.oid = x.indrelid
JOIN pg_class i ON i.oid = x.indexrelid
JOIN pg_namespace n ON n.oid = t.relnamespace
WHERE n.nspname = 'public'
AND t.relname IN (
'riskalerts', 'belt_allocation', 'portfolio_branch', 'main',
'main_operations', 'temp_reporting'
)
ORDER BY t.relname, i.relname;
SELECT
relname,
n_live_tup,
n_dead_tup,
seq_scan,
seq_tup_read,
idx_scan,
pg_size_pretty(pg_total_relation_size(relid)) AS total_size
FROM pg_stat_user_tables
WHERE relname IN (
'riskalerts', 'belt_allocation', 'portfolio_branch', 'main',
'main_operations', 'temp_reporting'
)
ORDER BY pg_total_relation_size(relid) DESC;
-- ---------------------------------------------------------------------------
-- Phase 1: high-confidence join and filter indexes
-- ---------------------------------------------------------------------------
-- visit_mis/current_visit_mis join riskalerts using both columns. The dump has
-- separate uuid and visit indexes, which do not provide this composite lookup.
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_riskalerts_uuid_visit
ON public.riskalerts USING btree (uuid, visit);
-- colony_allocation is a view joining colony to belt_allocation by belt_id.
-- Only belt_allocation(verifier_id) exists in the supplied schema.
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_belt_allocation_belt_verifier
ON public.belt_allocation USING btree (belt_id, verifier_id);
-- portfolio_bank_branch expands through portfolio_branch. These support its
-- joins and the common portfolio/isactive lookup used by nimble.qry.
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_portfolio_branch_portfolio_active
ON public.portfolio_branch USING btree (portfolio_id, isactive, portbranch_id);
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_portfolio_branch_bank_branch
ON public.portfolio_branch USING btree (bank_branch_id);
-- Dashboard hourly_tobesolved_cases filters on tobesolvedon and then reads
-- sdone/sdoneon. The existing temp_reporting indexes cover uuid and isclosed.
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_temp_reporting_tobesolvedon_open
ON public.temp_reporting USING btree (tobesolvedon)
INCLUDE (sdone, sdoneon, portfolio_id)
WHERE tobesolvedon IS NOT NULL;
-- scan_mis and Dashboard read today's completed scans. This avoids scanning all
-- main_operations rows having the same low-cardinality sdone value.
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_main_operations_scan_doneon
ON public.main_operations USING btree (sdoneon, sdoneby, uuid)
WHERE sdone = 1 AND sdoneon IS NOT NULL;
-- Visit MIS expands residence/office/property separately. These partial indexes
-- remove deleted/non-applicable rows before branch/verifier matching. They also
-- retain uuid for the main_operations join.
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_main_visit_res_branch_verifier
ON public.main USING btree (branch_id, resiverifier, uuid)
WHERE isdeleted = 0 AND rv = 1;
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_main_visit_off_branch_verifier
ON public.main USING btree (branch_id, offverifier, uuid)
WHERE isdeleted = 0 AND ov = 1;
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_main_visit_prop_branch_verifier
ON public.main USING btree (branch_id, propverifier, uuid)
WHERE isdeleted = 0 AND pv = 1;
-- vervisitmis queries verifier_monthly_data by verifier and receivedate. These
-- are partial and therefore smaller than unrestricted composite indexes.
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_main_monthly_res_verifier_date
ON public.main USING btree (resiverifier, receivedate)
WHERE isdeleted = 0 AND rv = 1 AND sradd = 0;
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_main_monthly_off_verifier_date
ON public.main USING btree (offverifier, receivedate)
WHERE isdeleted = 0 AND ov = 1 AND soadd = 0;
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_main_monthly_prop_verifier_date
ON public.main USING btree (propverifier, receivedate)
WHERE isdeleted = 0 AND pv = 1 AND spadd = 0;
-- ---------------------------------------------------------------------------
-- Phase 2: foreign-key support indexes
-- ---------------------------------------------------------------------------
-- These do not directly solve Visit MIS/Dashboard latency. They prevent child
-- table scans during parent updates/deletes and support related joins.
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_case_documents_uuid
ON public.case_documents USING btree (uuid);
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_contact_mapping_contact_id
ON public.contact_mapping USING btree (contact_id);
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_contacts_history_contact_id
ON public.contacts_history USING btree (contact_id);
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_document_data_uuid
ON public.document_data USING btree (uuid);
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_field_mapping_field_id
ON public.field_mapping USING btree (field_id);
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_field_mapping_template_id
ON public.field_mapping USING btree (template_id);
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_field_value_field_id
ON public.field_value USING btree (field_id);
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_onlinecasedetails_uuid
ON public.onlinecasedetails USING btree (uuid);
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_options_mapping_opt_value_id
ON public.options_mapping USING btree (opt_value_id);
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_permission_page_id
ON public.permission USING btree (page_id);
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_phoneno_history_uuid
ON public.phoneno_history USING btree (uuid);
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_tagdoc_history_docid
ON public.tagdoc_history USING btree (docid);
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_tagged_docs_uuid
ON public.tagged_docs USING btree (uuid);
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_template_mapping_portfolio_id
ON public.template_mapping USING btree (portfolio_id);
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_template_mapping_template_id
ON public.template_mapping USING btree (template_id);
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_tracker_template_mapping_portfolio
ON public.tracker_template_mapping USING btree (portfolio_id);
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_tracker_template_mapping_tracker
ON public.tracker_template_mapping USING btree (tracker_id);
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_tracker_sheet_layout_sheet
ON public.tracker_template_sheet_layout USING btree (sheet_id);
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_tracker_template_sheets_tracker
ON public.tracker_template_sheets USING btree (tracker_id);
-- Refresh planner statistics after all builds. ANALYZE does not rewrite tables,
-- but it consumes resources, so run it after the index builds at low traffic.
ANALYZE public.riskalerts;
ANALYZE public.belt_allocation;
ANALYZE public.portfolio_branch;
ANALYZE public.temp_reporting;
ANALYZE public.main_operations;
ANALYZE public.main;
-- ---------------------------------------------------------------------------
-- Postflight validation
-- ---------------------------------------------------------------------------
SELECT
indexrelname,
idx_scan,
idx_tup_read,
idx_tup_fetch,
pg_size_pretty(pg_relation_size(indexrelid)) AS index_size
FROM pg_stat_user_indexes
WHERE indexrelname LIKE 'idx_%'
AND relname IN (
'riskalerts', 'belt_allocation', 'portfolio_branch', 'main',
'main_operations', 'temp_reporting'
)
ORDER BY relname, indexrelname;
-- Check for invalid indexes left by an interrupted concurrent build.
SELECT n.nspname, c.relname AS index_name
FROM pg_index i
JOIN pg_class c ON c.oid = i.indexrelid
JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE n.nspname = 'public'
AND NOT i.indisvalid;
-- ---------------------------------------------------------------------------
-- Emergency rollback reference -- intentionally commented out
-- ---------------------------------------------------------------------------
-- Uncomment and run only the index that must be removed. Do not execute this
-- section after a successful migration unless a measured regression occurs.
--
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_riskalerts_uuid_visit;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_belt_allocation_belt_verifier;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_portfolio_branch_portfolio_active;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_portfolio_branch_bank_branch;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_temp_reporting_tobesolvedon_open;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_main_operations_scan_doneon;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_main_visit_res_branch_verifier;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_main_visit_off_branch_verifier;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_main_visit_prop_branch_verifier;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_main_monthly_res_verifier_date;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_main_monthly_off_verifier_date;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_main_monthly_prop_verifier_date;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_case_documents_uuid;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_contact_mapping_contact_id;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_contacts_history_contact_id;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_document_data_uuid;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_field_mapping_field_id;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_field_mapping_template_id;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_field_value_field_id;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_onlinecasedetails_uuid;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_options_mapping_opt_value_id;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_permission_page_id;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_phoneno_history_uuid;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_tagdoc_history_docid;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_tagged_docs_uuid;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_template_mapping_portfolio_id;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_template_mapping_template_id;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_tracker_template_mapping_portfolio;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_tracker_template_mapping_tracker;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_tracker_sheet_layout_sheet;
-- DROP INDEX CONCURRENTLY IF EXISTS public.idx_tracker_template_sheets_tracker;