Government publications. Part of what the twin can answer. Every figure below is the value the route serves, with its source; the raw response is at /api/twin/brain/govfull.
Descriptive, sourced: no model
Basis: sourced records (URL, sha256 or locator per row), no model.
One row per stored full government document; issuing agency is mapped to registry entities; overlap with objectstorage is determined by SHA-256; samples and pilot are measured, the remainder is counted, never estimated into served values except the labelled projection.
| Agency | Portal | Files | Files also in the office store |
|---|---|---|---|
| Implementation Monitoring and Evaluation Division | Portal: imed.gov.bd | Files: 755 | Files also in the office store: 12 |
| Bangladesh Bank |
| Verdict | PDFs |
|---|---|
| Text | PDFs: 352 |
| Legacy bangla |
cold runs only: the verify random sample (read and hash rate), the deterministic text-check sample, and the pilot chunk run; text-layer PDFs estimated from the 400-file text-check sample's text share
| Source | Publisher | Unit | Caveat |
|---|---|---|---|
| Government untruncated corpus harvest (stored copies, gap row A3) | Publisher: Issuing ministries, departments, divisions and boards of the Government of Bangladesh (stored copies) | Unit: one row per stored PDF document: manifest line, URL, sha256 at fetch, bytes, chars, issuing agency, portal host, upload year/month, registry entity binding, overlap with the stored files from government portals; source file is relative to the corpus root | Caveat: All 4,049 documents are full PDF publications; 241 match SHA-256 hashes of the stored files from government portals (BRN46). Agency assignment is verified against exact registry entities. sha256 verified is set only for re-hashed files (the 400 deterministic random sample and the pilot); the rest carry the harvest sha256. |
phones_detected), never stored.| Portal: bb.org.bd |
| Files: 547 |
| Files also in the office store: 0 |
| Ministry of Finance (Finance Division) | Portal: mof.gov.bd | Files: 535 | Files also in the office store: 12 |
| Bangladesh Bureau of Statistics | Portal: bbs.gov.bd | Files: 439 | Files also in the office store: 1 |
| National Board of Revenue | Portal: nbr.gov.bd | Files: 334 | Files also in the office store: 14 |
| Directorate General of Health Services | Portal: dghs.gov.bd | Files: 295 | Files also in the office store: 60 |
| Department of Environment | Portal: doe.gov.bd | Files: 139 | Files also in the office store: 10 |
| Bangladesh Power Development Board | Portal: bpdb.gov.bd | Files: 133 | Files also in the office store: 5 |
| Department of Social Services | Portal: dss.gov.bd | Files: 100 | Files also in the office store: 17 |
| Export Promotion Bureau | Portal: epb.gov.bd | Files: 67 | Files also in the office store: 7 |
| Department of Agricultural Marketing | Portal: dam.gov.bd | Files: 66 | Files also in the office store: 2 |
| Ministry of Food | Portal: mofood.gov.bd | Files: 53 | Files also in the office store: 7 |
| Department of Agricultural Extension | Portal: dae.gov.bd | Files: 52 | Files also in the office store: 8 |
| Planning Commission | Portal: plancomm.gov.bd | Files: 51 | Files also in the office store: 8 |
| Directorate of Primary Education | Portal: dpe.gov.bd | Files: 46 | Files also in the office store: 3 |
| Economic Relations Division | Portal: erd.gov.bd | Files: 46 | Files also in the office store: 1 |
| Ministry of Commerce | Portal: mincom.gov.bd | Files: 46 | Files also in the office store: 6 |
| Directorate of Secondary and Higher Education | Portal: dshe.gov.bd | Files: 41 | Files also in the office store: 8 |
| Local Government Engineering Department | Portal: lged.gov.bd | Files: 39 | Files also in the office store: 7 |
| Power Division | Portal: powerdivision.gov.bd | Files: 38 | Files also in the office store: 10 |
| Bureau of Manpower, Employment and Training | Portal: bmet.gov.bd | Files: 33 | Files also in the office store: 1 |
| National Portal | Portal: bangladesh.gov.bd | Files: 33 | Files also in the office store: 15 |
| Sustainable and Renewable Energy Development Authority | Portal: sreda.gov.bd | Files: 30 | Files also in the office store: 6 |
| Petrobangla | Portal: petrobangla.org.bd | Files: 28 | Files also in the office store: 9 |
| Ministry of Public Administration | Portal: mopa.gov.bd | Files: 27 | Files also in the office store: 6 |
| Bangladesh Telecommunication Regulatory Commission | Portal: btrc.gov.bd | Files: 18 | Files also in the office store: 5 |
| Bangladesh Export Processing Zones Authority | Portal: bepza.gov.bd | Files: 17 | Files also in the office store: 0 |
| Bangladesh Petroleum Corporation | Portal: bpc.gov.bd | Files: 14 | Files also in the office store: 1 |
| Comptroller and Auditor General | Portal: cagbd.org | Files: 13 | Files also in the office store: 0 |
| Bangladesh Meteorological Department | Portal: bmd.gov.bd | Files: 5 | Files also in the office store: 0 |
| Dhaka Stock Exchange | Portal: dsebd.org | Files: 4 | Files also in the office store: 0 |
| Bangladesh Bureau of Educational Information and Statistics | Portal: banbeis.gov.bd | Files: 2 | Files also in the office store: 0 |
| Bangladesh Election Commission | Portal: ecs.gov.bd | Files: 2 | Files also in the office store: 0 |
| Bangladesh Securities and Exchange Commission | Portal: sec.gov.bd | Files: 1 | Files also in the office store: 0 |
Implementation Monitoring and Evaluation Division · Bangladesh Bank · Ministry of Finance (Finance Division) · Bangladesh Bureau of Statistics · National Board of Revenue · Directorate General of Health Services · Department of Environment · Bangladesh Power Development Board · Department of Social Services · Export Promotion Bureau · Department of Agricultural Marketing · Ministry of Food · Department of Agricultural Extension · Planning Commission · Directorate of Primary Education · Economic Relations Division · Ministry of Commerce · Directorate of Secondary and Higher Education · Local Government Engineering Department · Power Division · Bureau of Manpower, Employment and Training · National Portal · Sustainable and Renewable Energy Development Authority · Petrobangla · Ministry of Public Administration · Bangladesh Telecommunication Regulatory Commission · Bangladesh Export Processing Zones Authority · Bangladesh Petroleum Corporation · Comptroller and Auditor General · Bangladesh Meteorological Department · Dhaka Stock Exchange · Bangladesh Bureau of Educational Information and Statistics · Bangladesh Election Commission · Bangladesh Securities and Exchange Commission · Portal office documents · Portals · Field presence · Unions · Local government
| Undecodable | PDFs: 8 |
| Kind | Documents |
|---|---|
| Quarterly report | Documents: 1 |
| Policy | Documents: 1 |
| Budget | Documents: 1 |
| Annual report | Documents: 1 |
| Report | Documents: 1 |
| Check | Documents |
|---|---|
| None | Documents: 12 |
| Forward host | Documents: 1 |
| Other gov domain | Documents: 1 |
| Agency | Pages | Chunks | Printed domain | Printed kind | Printed date | Printed memo |
|---|---|---|---|---|---|---|
| Bangladesh Bank | Pages: 19 | Chunks: 16 | Printed domain: not available | Printed kind: Quarterly Report | Printed date: not available | Printed memo: not available |
| Bangladesh Bureau of Statistics | Pages: 25 | Chunks: 14 | Printed domain: not available | Printed kind: policy | Printed date: not available | Printed memo: not available |
| Bangladesh Export Processing Zones Authority | Pages: 15 | Chunks: 14 | Printed domain: not available | Printed kind: not available | Printed date: not available | Printed memo: not available |
| Bangladesh Meteorological Department | Pages: 19 | Chunks: 18 | Printed domain: not available | Printed kind: not available | Printed date: not available | Printed memo: not available |
| Bangladesh Power Development Board | Pages: 24 | Chunks: 16 | Printed domain: not available | Printed kind: not available | Printed date: not available | Printed memo: not available |
| Bangladesh Telecommunication Regulatory Commission | Pages: 30 | Chunks: 16 | Printed domain: not available | Printed kind: বাজেট | Printed date: not available | Printed memo: not available |
| Department of Agricultural Extension | Pages: 24 | Chunks: 22 | Printed domain: www.dae.gov.bd | Printed kind: not available | Printed date: not available | Printed memo: not available |
| Department of Agricultural Marketing | Pages: 9 | Chunks: 8 | Printed domain: www.dam.portal.gov.bd | Printed kind: not available | Printed date: ২২/০৬/২০২৩ | Printed memo: ১২.০২.০০০০.০১৬.০৬.০০১.২৩.১৯৩ |
| Directorate General of Health Services | Pages: 26 | Chunks: 16 | Printed domain: not available | Printed kind: not available | Printed date: not available | Printed memo: not available |
| Department of Environment | Pages: 21 | Chunks: 7 | Printed domain: not available | Printed kind: not available | Printed date: not available | Printed memo: not available |
| Department of Social Services | Pages: 27 | Chunks: 12 | Printed domain: not available | Printed kind: not available | Printed date: not available | Printed memo: not available |
| Export Promotion Bureau | Pages: 17 | Chunks: 6 | Printed domain: not available | Printed kind: not available | Printed date: not available | Printed memo: not available |
| Economic Relations Division | Pages: 22 | Chunks: 21 | Printed domain: not available | Printed kind: Annual Report | Printed date: not available | Printed memo: not available |
| Implementation Monitoring and Evaluation Division | Pages: 15 | Chunks: 14 | Printed domain: not available | Printed kind: প্রতিবেদন | Printed date: not available | Printed memo: not available |
| Stage | Items | Seconds | Workers | Rate | Rate unit |
|---|---|---|---|---|---|
| Read | Items: 400 | Seconds: 1.203 | Workers: 4 | Rate: 1,692,653,623.441 | Rate unit: bytes read and hashed per second |
| Textcheck | Items: 400 | Seconds: 5.247 | Workers: 4 | Rate: 76.234 | Rate unit: PDFs text-checked per second |
| Chunk | Items: 154 | Seconds: 21.124 | Workers: 1 | Rate: 7.29 | Rate unit: candidate documents examined per second |
| Per-issuing-agency manifest datasets | Publisher: Built from the rows above by the inventory pipeline | Unit: one row per issuing agency, with its partition file, sha256, file counts, and text layer counts | Caveat: Each agency's inventory partition is the resumable unit of the pipeline; a rerun rewrites only partitions whose rows changed. |
| 200-chunk pilot: documents and chunks with printed fields across 14 agencies | Publisher: each document's stored PDF (PyMuPDF text layer) | Unit: one row per pilot document (fields with page and line) and per chunk (page range, counts, text sha256) | Caveat: Pilot documents were chosen to maximize agency diversity across text-layer candidates, chunked in consecutive page blocks up to 4,000 characters, hitting exactly 200 chunks total. Documents with personal data (CVs, data sheets, rosters, beneficiary lists, or >=2 mobile phone numbers on pages 1-2) are screened out before chunking. Phone numbers are never stored in any parquet or served payload. |
| Pipeline run log (measured) | Publisher: Measured pipeline runs on a 16-core machine | Unit: one row per stage run: items, bytes, seconds, workers | Caveat: Runs over files a previous run had read hit the page cache and overstate the rate; the projection uses the cold runs named in its basis. |