You kick off a bulk duplicate detection job against your Account or Contact table. You want a real number: how many duplicates are actually sitting in this environment. The job runs, finishes, and hands you a result set.
It stops at 5,000.
If your table has more duplicates than that, the job doesn't tell you it gave up early. It just returns what it found up to the ceiling, and the result looks complete unless you already know to be suspicious of it.
This isn't a bug or a misconfiguration. It's documented behavior. Every duplicate detection job, whether you launch it from Settings > Data Management > Duplicate Detection Jobs or trigger one through the API, runs on the same underlying mechanism (BulkDetectDuplicates), and Microsoft's own documentation states plainly that a duplicate detection job returns a maximum of 5,000 duplicate records.
It's also worth knowing that Dynamics 365 CE only ships default duplicate detection rules for accounts, contacts, and leads. Any other table needs a rule built and published before a bulk job will find anything on it at all.
Put those two facts together and the pattern is predictable: the tables most likely to have accumulated years of duplicates through imports, integrations, and manual entry (accounts and contacts) are also the ones where a single cleanup job is most likely to hit the 5,000 mark before it's covered the full dataset.
The Duplicate Detection Job wizard lets you scope a run with a saved view or a Select filter instead of running it against the whole table. That's the lever most admins reach for once they realize the ceiling exists:
It works. It's also entirely manual, and the bookkeeping is on you: a spreadsheet of which slice covered which date range, run on whatever cadence someone remembers to kick it off again.
A batched bulk job gives you a number for the day you ran it. It says nothing about the records created an hour later.
Every channel that doesn't go through the native duplicate detection dialog, Web API inserts, Power Automate flows, Dual-Write syncs, is still writing new records the whole time you're working through your batches. (We've written separately about how Web API inserts bypass duplicate detection by default, which is worth a read if integrations are a big part of how records land in your environment.) A bulk job, however carefully batched, is a snapshot. It tells you where things stood, not where they stand.
That's fine as an occasional audit. It's not a process you'd want to run your data quality on. The batching overhead scales with data volume, not down from it, so the bigger the environment gets, the more manual tracking the workaround demands, right as it becomes least practical to keep up with by hand.
A batched bulk job only ever solves half of this. It gets you caught up once (Clean), but it does nothing about the records created the moment after the job finishes (Prevent). Run one without the other and you're not closing the backlog, you're re-opening the same batching exercise on a recurring schedule.
Plauti for Dynamics 365 CE runs both inside your own Dataverse environment, installed from Microsoft Marketplace. Bulk detection has no 5,000-record ceiling and covers standard and custom entities, not just the three Microsoft ships default rules for, and the same rules check records arriving through the API, so integrations stop adding to the pile while you're clearing it. If you're mid-cleanup and want a number that doesn't depend on how many batches you remembered to run: plauti.com/crm/microsoft-dynamics.
Blog
Web API calls into Dataverse skip duplicate detection unless the request explicitly opts in. Here's why Dual-Write, KingswaySoft, and Power Automate hit this gap, and which fixes actually hold up.
Blog
Dynamics 365's native duplicate detection caps at 5,000 records per job and can't fuzzy-match. Here's how enterprises clean and prevent duplicates across 500,000+ records and business units.
Only tables with a published duplicate detection rule are scanned at all, and Dynamics 365 CE only ships default rules for accounts, contacts, and leads. On any of those tables, or a custom one with a rule you've built, a single bulk job still won't return more than 5,000 duplicates.
Not that Microsoft's documentation says. The job returns what it found up to 5,000 and finishes normally, so there's no built-in signal telling you whether that's the full picture or a partial one.
No. It's a fixed limit on the underlying BulkDetectDuplicates mechanism, not a configurable setting, so there's no admin option that changes it.
No. Whether it's a single record flagged on save or a full list from a bulk job, Dynamics 365 CE only identifies matches; it doesn't combine them. You still have to open each pair and use the native Merge function to choose which record survives.
Configured once per environment, covering every entry point. See how other D365 CE users restructured this.