Government publications by agency. Part of what the twin can answer. Every figure below is the value the route serves, with its source; the raw response is at /api/twin/brain/govfull/agency/bmet.
Descriptive, sourced: no model
Basis: sourced records (URL, sha256 or locator per row), no model.
stored files, documents and chunks, and counts over them
| Upload year | Files | Bytes |
|---|---|---|
| 2024 | Files: 30 | Bytes: 54,084,847 |
| 2026 | Files: 3 | Bytes: 2,375,942 |
| Verdict | PDFs |
|---|---|
| Text | PDFs: 2 |
| Manifest line | Type | Bytes | Characters | Upload year | Upload month | Also in the office store | Text layer | Pages | Re-hash verified |
|---|---|---|---|---|---|---|---|---|---|
| 174 | Type: pdf |
| Source | Publisher | Unit | Caveat |
|---|---|---|---|
| Government untruncated corpus harvest (stored copies, gap row A3) | Publisher: Issuing ministries, departments, divisions and boards of the Government of Bangladesh (stored copies) | Unit: one row per stored PDF document: manifest line, URL, sha256 at fetch, bytes, chars, issuing agency, portal host, upload year/month, registry entity binding, overlap with the stored files from government portals; source file is relative to the corpus root | Caveat: All 4,049 documents are full PDF publications; 241 match SHA-256 hashes of the stored files from government portals (BRN46). Agency assignment is verified against exact registry entities. sha256 verified is set only for re-hashed files (the 400 deterministic random sample and the pilot); the rest carry the harvest sha256. |
phones_detected), never stored.| Characters: 192,032 |
| Upload year: 2024 |
| Upload month: 12 |
| Also in the office store: no |
| Text layer: not available |
| Pages: not available |
| Re-hash verified: not available |
| 230 | Type: pdf | Bytes: 199,402 | Characters: 653,934 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 280 | Type: pdf | Bytes: 549,546 | Characters: 70,992 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 316 | Type: pdf | Bytes: 2,436,245 | Characters: 388,729 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 386 | Type: pdf | Bytes: 714,384 | Characters: 137,724 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 823 | Type: pdf | Bytes: 863,584 | Characters: 118,337 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 949 | Type: pdf | Bytes: 1,645,406 | Characters: 256,946 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,029 | Type: pdf | Bytes: 1,369,750 | Characters: 87,631 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,088 | Type: pdf | Bytes: 1,350,789 | Characters: 198,655 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,097 | Type: pdf | Bytes: 12,301,175 | Characters: 185,545 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,173 | Type: pdf | Bytes: 1,167,925 | Characters: 222,906 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,394 | Type: pdf | Bytes: 1,555,307 | Characters: 239,121 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,406 | Type: pdf | Bytes: 712,834 | Characters: 40,138 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,476 | Type: pdf | Bytes: 488,177 | Characters: 55,618 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,577 | Type: pdf | Bytes: 1,121,707 | Characters: 89,042 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,663 | Type: pdf | Bytes: 1,235,278 | Characters: 87,513 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,822 | Type: pdf | Bytes: 5,303,573 | Characters: 267,084 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,143 | Type: pdf | Bytes: 1,035,008 | Characters: 98,377 | Upload year: 2026 | Upload month: 8 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,148 | Type: pdf | Bytes: 587,227 | Characters: 74,222 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,420 | Type: pdf | Bytes: 9,107,123 | Characters: 128,862 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,450 | Type: pdf | Bytes: 608,122 | Characters: 89,857 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,548 | Type: pdf | Bytes: 1,301,919 | Characters: 201,581 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,704 | Type: pdf | Bytes: 951,041 | Characters: 133,892 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 33 | Re-hash verified: yes |
| 2,813 | Type: pdf | Bytes: 846,417 | Characters: 107,948 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,184 | Type: pdf | Bytes: 685,729 | Characters: 95,138 | Upload year: 2026 | Upload month: 0 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,198 | Type: pdf | Bytes: 1,822,744 | Characters: 283,105 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,242 | Type: pdf | Bytes: 369,739 | Characters: 113,035 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,404 | Type: pdf | Bytes: 414,566 | Characters: 117,131 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,606 | Type: pdf | Bytes: 225,102 | Characters: 100,821 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,680 | Type: pdf | Bytes: 331,446 | Characters: 510,837 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,734 | Type: pdf | Bytes: 2,584,400 | Characters: 355,206 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,902 | Type: pdf | Bytes: 655,205 | Characters: 95,436 | Upload year: 2026 | Upload month: 0 | Also in the office store: no | Text layer: Text | Pages: 109 | Re-hash verified: yes |
| 4,033 | Type: pdf | Bytes: 638,747 | Characters: 83,439 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| Per-issuing-agency manifest datasets | Publisher: Built from the rows above by the inventory pipeline | Unit: one row per issuing agency, with its partition file, sha256, file counts, and text layer counts | Caveat: Each agency's inventory partition is the resumable unit of the pipeline; a rerun rewrites only partitions whose rows changed. |
| 200-chunk pilot: documents and chunks with printed fields across 14 agencies | Publisher: each document's stored PDF (PyMuPDF text layer) | Unit: one row per pilot document (fields with page and line) and per chunk (page range, counts, text sha256) | Caveat: Pilot documents were chosen to maximize agency diversity across text-layer candidates, chunked in consecutive page blocks up to 4,000 characters, hitting exactly 200 chunks total. Documents with personal data (CVs, data sheets, rosters, beneficiary lists, or >=2 mobile phone numbers on pages 1-2) are screened out before chunking. Phone numbers are never stored in any parquet or served payload. |
| Pipeline run log (measured) | Publisher: Measured pipeline runs on a 16-core machine | Unit: one row per stage run: items, bytes, seconds, workers | Caveat: Runs over files a previous run had read hit the page cache and overstate the rate; the projection uses the cold runs named in its basis. |