Government publications by agency. Part of what the twin can answer. Every figure below is the value the route serves, with its source; the raw response is at /api/twin/brain/govfull/agency/cag.
Descriptive, sourced: no model
Basis: sourced records (URL, sha256 or locator per row), no model.
stored files, documents and chunks, and counts over them
| Verdict | PDFs |
|---|---|
| Text | PDFs: 1 |
| Manifest line | Type | Bytes | Characters | Also in the office store | Text layer | Pages | Re-hash verified |
|---|---|---|---|---|---|---|---|
| 334 | Type: pdf | Bytes: 1,203,310 |
| Source | Publisher | Unit | Caveat |
|---|---|---|---|
| Government untruncated corpus harvest (stored copies, gap row A3) | Publisher: Issuing ministries, departments, divisions and boards of the Government of Bangladesh (stored copies) | Unit: one row per stored PDF document: manifest line, URL, sha256 at fetch, bytes, chars, issuing agency, portal host, upload year/month, registry entity binding, overlap with the stored files from government portals; source file is relative to the corpus root | Caveat: All 4,049 documents are full PDF publications; 241 match SHA-256 hashes of the stored files from government portals (BRN46). Agency assignment is verified against exact registry entities. sha256 verified is set only for re-hashed files (the 400 deterministic random sample and the pilot); the rest carry the harvest sha256. |
phones_detected), never stored.| Also in the office store: no |
| Text layer: not available |
| Pages: not available |
| Re-hash verified: not available |
| 376 | Type: pdf | Bytes: 1,784,933 | Characters: 35,729 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 593 | Type: pdf | Bytes: 2,267,583 | Characters: 253,208 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 657 | Type: pdf | Bytes: 1,203,554 | Characters: 223,511 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 771 | Type: pdf | Bytes: 21,727,584 | Characters: 964,305 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,095 | Type: pdf | Bytes: 305,141 | Characters: 59,673 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,789 | Type: pdf | Bytes: 4,591,469 | Characters: 375,919 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,060 | Type: pdf | Bytes: 469,216 | Characters: 94,402 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,457 | Type: pdf | Bytes: 56,535,260 | Characters: 804,005 | Also in the office store: no | Text layer: Text | Pages: 421 | Re-hash verified: yes |
| 3,162 | Type: pdf | Bytes: 1,934,912 | Characters: 719,162 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,553 | Type: pdf | Bytes: 949,997 | Characters: 330,111 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,632 | Type: pdf | Bytes: 8,967,296 | Characters: 1,193,162 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 4,017 | Type: pdf | Bytes: 3,681,282 | Characters: 295,644 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
Not held: Upload year, Upload month.
| Per-issuing-agency manifest datasets | Publisher: Built from the rows above by the inventory pipeline | Unit: one row per issuing agency, with its partition file, sha256, file counts, and text layer counts | Caveat: Each agency's inventory partition is the resumable unit of the pipeline; a rerun rewrites only partitions whose rows changed. |
| 200-chunk pilot: documents and chunks with printed fields across 14 agencies | Publisher: each document's stored PDF (PyMuPDF text layer) | Unit: one row per pilot document (fields with page and line) and per chunk (page range, counts, text sha256) | Caveat: Pilot documents were chosen to maximize agency diversity across text-layer candidates, chunked in consecutive page blocks up to 4,000 characters, hitting exactly 200 chunks total. Documents with personal data (CVs, data sheets, rosters, beneficiary lists, or >=2 mobile phone numbers on pages 1-2) are screened out before chunking. Phone numbers are never stored in any parquet or served payload. |
| Pipeline run log (measured) | Publisher: Measured pipeline runs on a 16-core machine | Unit: one row per stage run: items, bytes, seconds, workers | Caveat: Runs over files a previous run had read hit the page cache and overstate the rate; the projection uses the cold runs named in its basis. |