Government publications by agency. Part of what the twin can answer. Every figure below is the value the route serves, with its source; the raw response is at /api/twin/brain/govfull/agency/nbr.
Descriptive, sourced: no model
Basis: sourced records (URL, sha256 or locator per row), no model.
stored files, documents and chunks, and counts over them
| Verdict | PDFs |
|---|---|
| Text | PDFs: 20 |
| Legacy bangla | PDFs: 10 |
| Undecodable | PDFs: 1 |
| Manifest line | Type | Bytes | Characters | Also in the office store | Text layer | Pages | Re-hash verified |
|---|---|---|---|---|---|---|---|
| 3 | Type: pdf | Bytes: 9,794,098 |
| Source | Publisher | Unit | Caveat |
|---|---|---|---|
| Government untruncated corpus harvest (stored copies, gap row A3) | Publisher: Issuing ministries, departments, divisions and boards of the Government of Bangladesh (stored copies) | Unit: one row per stored PDF document: manifest line, URL, sha256 at fetch, bytes, chars, issuing agency, portal host, upload year/month, registry entity binding, overlap with the stored files from government portals; source file is relative to the corpus root | Caveat: All 4,049 documents are full PDF publications; 241 match SHA-256 hashes of the stored files from government portals (BRN46). Agency assignment is verified against exact registry entities. sha256 verified is set only for re-hashed files (the 400 deterministic random sample and the pilot); the rest carry the harvest sha256. |
phones_detected), never stored.| Also in the office store: no |
| Text layer: not available |
| Pages: not available |
| Re-hash verified: not available |
| 6 | Type: pdf | Bytes: 588,713 | Characters: 75,160 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 11 | Type: pdf | Bytes: 610,665 | Characters: 87,386 | Also in the office store: no | Text layer: Text | Pages: 15 | Re-hash verified: yes |
| 46 | Type: pdf | Bytes: 109,414 | Characters: 99,758 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 65 | Type: pdf | Bytes: 2,889,170 | Characters: 385,451 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 70 | Type: pdf | Bytes: 545,097 | Characters: 90,089 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 83 | Type: pdf | Bytes: 1,543,982 | Characters: 119,101 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 91 | Type: pdf | Bytes: 678,454 | Characters: 100,297 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 102 | Type: pdf | Bytes: 2,607,859 | Characters: 347,351 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 108 | Type: pdf | Bytes: 913,223 | Characters: 149,879 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 109 | Type: pdf | Bytes: 384,346 | Characters: 223,696 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 126 | Type: pdf | Bytes: 2,781,113 | Characters: 955,386 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 157 | Type: pdf | Bytes: 3,121,172 | Characters: 377,556 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 172 | Type: pdf | Bytes: 881,822 | Characters: 64,343 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 190 | Type: pdf | Bytes: 397,885 | Characters: 66,368 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 196 | Type: pdf | Bytes: 1,075,659 | Characters: 78,854 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 204 | Type: pdf | Bytes: 1,516,570 | Characters: 166,033 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 217 | Type: pdf | Bytes: 580,733 | Characters: 63,403 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 228 | Type: pdf | Bytes: 685,973 | Characters: 79,142 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 231 | Type: pdf | Bytes: 606,636 | Characters: 56,243 | Also in the office store: no | Text layer: Text | Pages: 23 | Re-hash verified: yes |
| 235 | Type: pdf | Bytes: 613,747 | Characters: 107,316 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 241 | Type: pdf | Bytes: 1,360,698 | Characters: 655,374 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 255 | Type: pdf | Bytes: 6,157,688 | Characters: 608,355 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 286 | Type: pdf | Bytes: 739,757 | Characters: 135,885 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 302 | Type: pdf | Bytes: 1,134,623 | Characters: 519,974 | Also in the office store: no | Text layer: Legacy bangla | Pages: 71 | Re-hash verified: yes |
| 305 | Type: pdf | Bytes: 1,310,834 | Characters: 352,519 | Also in the office store: no | Text layer: Legacy bangla | Pages: 110 | Re-hash verified: yes |
| 309 | Type: pdf | Bytes: 673,501 | Characters: 99,954 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 325 | Type: pdf | Bytes: 700,434 | Characters: 60,974 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 328 | Type: pdf | Bytes: 497,502 | Characters: 81,631 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 371 | Type: pdf | Bytes: 2,127,965 | Characters: 189,529 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 393 | Type: pdf | Bytes: 976,255 | Characters: 61,900 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 403 | Type: pdf | Bytes: 2,274,845 | Characters: 132,075 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 408 | Type: pdf | Bytes: 976,789 | Characters: 86,276 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 414 | Type: pdf | Bytes: 431,504 | Characters: 46,183 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 453 | Type: pdf | Bytes: 983,843 | Characters: 264,781 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 456 | Type: pdf | Bytes: 1,030,393 | Characters: 45,426 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 472 | Type: pdf | Bytes: 367,558 | Characters: 65,210 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 473 | Type: pdf | Bytes: 1,976,710 | Characters: 519,797 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 476 | Type: pdf | Bytes: 729,054 | Characters: 73,606 | Also in the office store: no | Text layer: Legacy bangla | Pages: 21 | Re-hash verified: yes |
| 479 | Type: pdf | Bytes: 2,315,573 | Characters: 222,657 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 491 | Type: pdf | Bytes: 610,286 | Characters: 87,036 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 493 | Type: pdf | Bytes: 2,533,655 | Characters: 197,853 | Also in the office store: no | Text layer: Text | Pages: 119 | Re-hash verified: yes |
| 503 | Type: pdf | Bytes: 1,097,431 | Characters: 302,927 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 509 | Type: pdf | Bytes: 822,558 | Characters: 61,157 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 524 | Type: pdf | Bytes: 968,329 | Characters: 294,263 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 527 | Type: pdf | Bytes: 5,636,405 | Characters: 68,129 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 530 | Type: pdf | Bytes: 556,824 | Characters: 78,682 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 531 | Type: pdf | Bytes: 2,326,632 | Characters: 331,029 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 542 | Type: pdf | Bytes: 4,581,088 | Characters: 75,582 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 548 | Type: pdf | Bytes: 1,172,799 | Characters: 95,560 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
Not held: Upload year, Upload month.
| Per-issuing-agency manifest datasets | Publisher: Built from the rows above by the inventory pipeline | Unit: one row per issuing agency, with its partition file, sha256, file counts, and text layer counts | Caveat: Each agency's inventory partition is the resumable unit of the pipeline; a rerun rewrites only partitions whose rows changed. |
| 200-chunk pilot: documents and chunks with printed fields across 14 agencies | Publisher: each document's stored PDF (PyMuPDF text layer) | Unit: one row per pilot document (fields with page and line) and per chunk (page range, counts, text sha256) | Caveat: Pilot documents were chosen to maximize agency diversity across text-layer candidates, chunked in consecutive page blocks up to 4,000 characters, hitting exactly 200 chunks total. Documents with personal data (CVs, data sheets, rosters, beneficiary lists, or >=2 mobile phone numbers on pages 1-2) are screened out before chunking. Phone numbers are never stored in any parquet or served payload. |
| Pipeline run log (measured) | Publisher: Measured pipeline runs on a 16-core machine | Unit: one row per stage run: items, bytes, seconds, workers | Caveat: Runs over files a previous run had read hit the page cache and overstate the rate; the projection uses the cold runs named in its basis. |