Government publications by agency. Part of what the twin can answer. Every figure below is the value the route serves, with its source; the raw response is at /api/twin/brain/govfull/agency/doe.
Descriptive, sourced: no model
Basis: sourced records (URL, sha256 or locator per row), no model.
stored files, documents and chunks, and counts over them
| Upload year | Files | Bytes |
|---|---|---|
| 2024 | Files: 118 | Bytes: 393,129,779 |
| 2026 | Files: 21 | Bytes: 87,202,676 |
| Verdict | PDFs |
|---|---|
| Text | PDFs: 14 |
| Legacy bangla | PDFs: 1 |
| Manifest line | Type | Bytes | Characters | Upload year | Upload month | Also in the office store | Text layer | Pages | Re-hash verified |
|---|---|---|---|---|---|---|---|---|---|
| 9 | Type: pdf |
| Pages | Chunks |
|---|---|
| 21 | Chunks: 7 |
Not held: Printed domain, Page, Line, Printed kind, Page, Line, Printed date, Page, Line, Printed memo, Page, Line.
| First page | Last page | Characters | Lines | Numeric tokens | Bangla share |
|---|---|---|---|---|---|
| 1 | Last page: 3 | Characters: 3,996 | Lines: 74 |
| Source | Publisher | Unit | Caveat |
|---|---|---|---|
| Government untruncated corpus harvest (stored copies, gap row A3) | Publisher: Issuing ministries, departments, divisions and boards of the Government of Bangladesh (stored copies) | Unit: one row per stored PDF document: manifest line, URL, sha256 at fetch, bytes, chars, issuing agency, portal host, upload year/month, registry entity binding, overlap with the stored files from government portals; source file is relative to the corpus root | Caveat: All 4,049 documents are full PDF publications; 241 match SHA-256 hashes of the stored files from government portals (BRN46). Agency assignment is verified against exact registry entities. sha256 verified is set only for re-hashed files (the 400 deterministic random sample and the pilot); the rest carry the harvest sha256. |
phones_detected), never stored.| Characters: 53,076 |
| Upload year: 2024 |
| Upload month: 12 |
| Also in the office store: yes |
| Text layer: not available |
| Pages: not available |
| Re-hash verified: not available |
| 24 | Type: pdf | Bytes: 14,397,018 | Characters: 468,052 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 60 | Type: pdf | Bytes: 976,178 | Characters: 71,227 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 84 | Type: pdf | Bytes: 672,202 | Characters: 172,725 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 110 | Type: pdf | Bytes: 604,855 | Characters: 74,275 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 120 | Type: pdf | Bytes: 1,896,256 | Characters: 71,553 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 138 | Type: pdf | Bytes: 372,929 | Characters: 64,975 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 143 | Type: pdf | Bytes: 954,338 | Characters: 162,362 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 168 | Type: pdf | Bytes: 1,151,807 | Characters: 192,757 | Upload year: 2026 | Upload month: 7 | Also in the office store: no | Text layer: Text | Pages: 47 | Re-hash verified: yes |
| 226 | Type: pdf | Bytes: 11,604,386 | Characters: 2,296,152 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 269 | Type: pdf | Bytes: 1,071,469 | Characters: 97,942 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 310 | Type: pdf | Bytes: 867,735 | Characters: 126,778 | Upload year: 2026 | Upload month: 3 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 335 | Type: pdf | Bytes: 1,494,686 | Characters: 127,976 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 28 | Re-hash verified: yes |
| 392 | Type: pdf | Bytes: 2,686,503 | Characters: 353,591 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 425 | Type: pdf | Bytes: 1,184,991 | Characters: 55,450 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 21 | Re-hash verified: yes |
| 512 | Type: pdf | Bytes: 1,294,863 | Characters: 63,577 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 513 | Type: pdf | Bytes: 925,350 | Characters: 155,282 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 522 | Type: pdf | Bytes: 2,954,290 | Characters: 85,374 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 577 | Type: pdf | Bytes: 3,816,014 | Characters: 291,081 | Upload year: 2026 | Upload month: 0 | Also in the office store: no | Text layer: Text | Pages: 166 | Re-hash verified: yes |
| 613 | Type: pdf | Bytes: 376,270 | Characters: 67,590 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 627 | Type: pdf | Bytes: 7,197,327 | Characters: 916,184 | Upload year: 2026 | Upload month: 6 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 639 | Type: pdf | Bytes: 1,057,286 | Characters: 227,676 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 708 | Type: pdf | Bytes: 5,658,505 | Characters: 861,710 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 710 | Type: pdf | Bytes: 12,141,584 | Characters: 2,367,526 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 733 | Type: pdf | Bytes: 1,371,867 | Characters: 121,233 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 764 | Type: pdf | Bytes: 1,711,265 | Characters: 181,764 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 766 | Type: pdf | Bytes: 376,270 | Characters: 67,590 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 801 | Type: pdf | Bytes: 493,195 | Characters: 72,348 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 833 | Type: pdf | Bytes: 672,202 | Characters: 172,725 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 842 | Type: pdf | Bytes: 4,481,176 | Characters: 189,469 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 877 | Type: pdf | Bytes: 8,512,939 | Characters: 405,331 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 888 | Type: pdf | Bytes: 331,716 | Characters: 33,593 | Upload year: 2026 | Upload month: 5 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 898 | Type: pdf | Bytes: 622,334 | Characters: 66,590 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 900 | Type: pdf | Bytes: 1,299,641 | Characters: 67,413 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 915 | Type: pdf | Bytes: 1,171,323 | Characters: 64,562 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 972 | Type: pdf | Bytes: 298,821 | Characters: 53,076 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 996 | Type: pdf | Bytes: 8,666,605 | Characters: 229,532 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,068 | Type: pdf | Bytes: 468,848 | Characters: 63,223 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,092 | Type: pdf | Bytes: 640,682 | Characters: 66,475 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,115 | Type: pdf | Bytes: 2,082,576 | Characters: 183,141 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,156 | Type: pdf | Bytes: 9,592,900 | Characters: 376,154 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,221 | Type: pdf | Bytes: 604,855 | Characters: 74,275 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,234 | Type: pdf | Bytes: 4,323,466 | Characters: 366,349 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,253 | Type: pdf | Bytes: 1,044,329 | Characters: 164,559 | Upload year: 2026 | Upload month: 4 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,292 | Type: pdf | Bytes: 627,885 | Characters: 65,725 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,305 | Type: pdf | Bytes: 11,890,802 | Characters: 1,627,474 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 458 | Re-hash verified: yes |
| 1,310 | Type: pdf | Bytes: 10,469,849 | Characters: 1,052,143 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,353 | Type: pdf | Bytes: 1,836,820 | Characters: 176,585 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,408 | Type: pdf | Bytes: 1,154,603 | Characters: 207,210 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,428 | Type: pdf | Bytes: 2,473,608 | Characters: 212,176 | Upload year: 2026 | Upload month: 1 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| Bangla share: 89.6% |
| 4 | Last page: 7 | Characters: 3,278 | Lines: 105 | Numeric tokens: 73 | Bangla share: 89.25% |
| 8 | Last page: 10 | Characters: 3,204 | Lines: 362 | Numeric tokens: 233 | Bangla share: 92.13% |
| 11 | Last page: 16 | Characters: 3,936 | Lines: 407 | Numeric tokens: 233 | Bangla share: 73.14% |
| 17 | Last page: 17 | Characters: 2,493 | Lines: 83 | Numeric tokens: 32 | Bangla share: 89.84% |
| 18 | Last page: 18 | Characters: 2,693 | Lines: 88 | Numeric tokens: 33 | Bangla share: 94.04% |
| 19 | Last page: 21 | Characters: 3,698 | Lines: 114 | Numeric tokens: 24 | Bangla share: 93.36% |
| Per-issuing-agency manifest datasets | Publisher: Built from the rows above by the inventory pipeline | Unit: one row per issuing agency, with its partition file, sha256, file counts, and text layer counts | Caveat: Each agency's inventory partition is the resumable unit of the pipeline; a rerun rewrites only partitions whose rows changed. |
| 200-chunk pilot: documents and chunks with printed fields across 14 agencies | Publisher: each document's stored PDF (PyMuPDF text layer) | Unit: one row per pilot document (fields with page and line) and per chunk (page range, counts, text sha256) | Caveat: Pilot documents were chosen to maximize agency diversity across text-layer candidates, chunked in consecutive page blocks up to 4,000 characters, hitting exactly 200 chunks total. Documents with personal data (CVs, data sheets, rosters, beneficiary lists, or >=2 mobile phone numbers on pages 1-2) are screened out before chunking. Phone numbers are never stored in any parquet or served payload. |
| Pipeline run log (measured) | Publisher: Measured pipeline runs on a 16-core machine | Unit: one row per stage run: items, bytes, seconds, workers | Caveat: Runs over files a previous run had read hit the page cache and overstate the rate; the projection uses the cold runs named in its basis. |