Government publications by agency. Part of what the twin can answer. Every figure below is the value the route serves, with its source; the raw response is at /api/twin/brain/govfull/agency/plancomm.
Descriptive, sourced: no model
Basis: sourced records (URL, sha256 or locator per row), no model.
stored files, documents and chunks, and counts over them
| Upload year | Files | Bytes |
|---|---|---|
| 2024 | Files: 45 | Bytes: 338,008,157 |
| 2026 | Files: 6 | Bytes: 53,005,556 |
| Verdict | PDFs |
|---|---|
| Text | PDFs: 2 |
| Manifest line | Type | Bytes | Characters | Upload year | Upload month | Also in the office store | Text layer | Pages | Re-hash verified |
|---|---|---|---|---|---|---|---|---|---|
| 98 | Type: pdf |
| Source | Publisher | Unit | Caveat |
|---|---|---|---|
| Government untruncated corpus harvest (stored copies, gap row A3) | Publisher: Issuing ministries, departments, divisions and boards of the Government of Bangladesh (stored copies) | Unit: one row per stored PDF document: manifest line, URL, sha256 at fetch, bytes, chars, issuing agency, portal host, upload year/month, registry entity binding, overlap with the stored files from government portals; source file is relative to the corpus root | Caveat: All 4,049 documents are full PDF publications; 241 match SHA-256 hashes of the stored files from government portals (BRN46). Agency assignment is verified against exact registry entities. sha256 verified is set only for re-hashed files (the 400 deterministic random sample and the pilot); the rest carry the harvest sha256. |
phones_detected), never stored.| Characters: 1,076,748 |
| Upload year: 2024 |
| Upload month: 12 |
| Also in the office store: no |
| Text layer: not available |
| Pages: not available |
| Re-hash verified: not available |
| 209 | Type: pdf | Bytes: 4,441,466 | Characters: 140,036 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 314 | Type: pdf | Bytes: 1,474,733 | Characters: 219,090 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 339 | Type: pdf | Bytes: 9,200,446 | Characters: 2,887,817 | Upload year: 2026 | Upload month: 7 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 355 | Type: pdf | Bytes: 1,426,113 | Characters: 706,750 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 200 | Re-hash verified: yes |
| 437 | Type: pdf | Bytes: 2,164,231 | Characters: 317,724 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 558 | Type: pdf | Bytes: 11,266,308 | Characters: 5,956,238 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 601 | Type: pdf | Bytes: 12,537,165 | Characters: 5,278,149 | Upload year: 2026 | Upload month: 0 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 645 | Type: pdf | Bytes: 25,596,621 | Characters: 632,760 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 647 | Type: pdf | Bytes: 894,071 | Characters: 351,056 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 668 | Type: pdf | Bytes: 7,594,458 | Characters: 376,475 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 923 | Type: pdf | Bytes: 5,795,019 | Characters: 188,072 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 924 | Type: pdf | Bytes: 805,798 | Characters: 120,453 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 37 | Re-hash verified: yes |
| 1,117 | Type: pdf | Bytes: 3,660,342 | Characters: 436,932 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,136 | Type: pdf | Bytes: 19,811,193 | Characters: 552,771 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,167 | Type: pdf | Bytes: 1,337,307 | Characters: 670,422 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,269 | Type: pdf | Bytes: 2,399,508 | Characters: 673,703 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,386 | Type: pdf | Bytes: 1,211,364 | Characters: 890,427 | Upload year: 2026 | Upload month: 7 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,421 | Type: pdf | Bytes: 16,793,089 | Characters: 274,543 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,452 | Type: pdf | Bytes: 1,039,750 | Characters: 444,415 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,489 | Type: pdf | Bytes: 1,803,097 | Characters: 163,105 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,838 | Type: pdf | Bytes: 12,913,982 | Characters: 7,644 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,963 | Type: pdf | Bytes: 3,236,643 | Characters: 806,640 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,999 | Type: pdf | Bytes: 9,200,446 | Characters: 2,887,817 | Upload year: 2026 | Upload month: 7 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,013 | Type: pdf | Bytes: 2,968,164 | Characters: 809,470 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,124 | Type: pdf | Bytes: 8,056,336 | Characters: 2,095,497 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,230 | Type: pdf | Bytes: 2,928,282 | Characters: 718,652 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,231 | Type: pdf | Bytes: 753,252 | Characters: 184,715 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,383 | Type: pdf | Bytes: 10,158,979 | Characters: 853,239 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,399 | Type: pdf | Bytes: 13,372,893 | Characters: 768,174 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,495 | Type: pdf | Bytes: 4,722,995 | Characters: 815,229 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,647 | Type: pdf | Bytes: 1,211,364 | Characters: 890,427 | Upload year: 2026 | Upload month: 7 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,665 | Type: pdf | Bytes: 3,358,458 | Characters: 379,472 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,754 | Type: pdf | Bytes: 26,004,999 | Characters: 537,182 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,773 | Type: pdf | Bytes: 2,498,672 | Characters: 445,739 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,799 | Type: pdf | Bytes: 20,130,470 | Characters: 2,444,479 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,840 | Type: pdf | Bytes: 3,497,151 | Characters: 501,435 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,993 | Type: pdf | Bytes: 16,045,063 | Characters: 1,111,639 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,135 | Type: pdf | Bytes: 19,644,771 | Characters: 5,041,814 | Upload year: 2026 | Upload month: 6 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,174 | Type: pdf | Bytes: 1,500,726 | Characters: 722,113 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,197 | Type: pdf | Bytes: 5,856,096 | Characters: 358,801 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,451 | Type: pdf | Bytes: 13,101,041 | Characters: 572,309 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,555 | Type: pdf | Bytes: 4,590,596 | Characters: 369,676 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,603 | Type: pdf | Bytes: 4,064,705 | Characters: 499,072 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,640 | Type: pdf | Bytes: 609,471 | Characters: 82,591 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,666 | Type: pdf | Bytes: 1,324,959 | Characters: 552,675 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,671 | Type: pdf | Bytes: 3,892,749 | Characters: 1,479,780 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,709 | Type: pdf | Bytes: 14,450,400 | Characters: 593,877 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,722 | Type: pdf | Bytes: 18,061,934 | Characters: 97,638 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,773 | Type: pdf | Bytes: 4,565,500 | Characters: 459,966 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| Per-issuing-agency manifest datasets | Publisher: Built from the rows above by the inventory pipeline | Unit: one row per issuing agency, with its partition file, sha256, file counts, and text layer counts | Caveat: Each agency's inventory partition is the resumable unit of the pipeline; a rerun rewrites only partitions whose rows changed. |
| 200-chunk pilot: documents and chunks with printed fields across 14 agencies | Publisher: each document's stored PDF (PyMuPDF text layer) | Unit: one row per pilot document (fields with page and line) and per chunk (page range, counts, text sha256) | Caveat: Pilot documents were chosen to maximize agency diversity across text-layer candidates, chunked in consecutive page blocks up to 4,000 characters, hitting exactly 200 chunks total. Documents with personal data (CVs, data sheets, rosters, beneficiary lists, or >=2 mobile phone numbers on pages 1-2) are screened out before chunking. Phone numbers are never stored in any parquet or served payload. |
| Pipeline run log (measured) | Publisher: Measured pipeline runs on a 16-core machine | Unit: one row per stage run: items, bytes, seconds, workers | Caveat: Runs over files a previous run had read hit the page cache and overstate the rate; the projection uses the cold runs named in its basis. |