Remove Duplicate Entries Using KnexJS | ATLASSC.NET
Skip to main content
/// .DATA.FRAGMENT_20221126

Remove Duplicate Entries Using KnexJS

1 minAuthor: Z.SHINCHVEN

I accidentally created duplicate entries in my database, so I wrote a function to remove them. The MySQl database connector I use frequently is KnexJS.

Here's how I did it:

const removeDuplicateEntries = async (knex: Knex) => {
  // find duplicate entries
  const duplicates = await knex<MediaModel>('medias')
    .select('filepath')
    .from('medias')
    .groupBy('filepath')
    .havingRaw('count(*) > 1');
  // remove duplicate entries
  for (const duplicate of duplicates) {
    if (duplicate.filepath && duplicate.filepath.length > 0) {
      // find the first entry of the duplicated record.
      const keep = await knex('medias')
        .select('id')
        .where('filepath', duplicate.filepath)
        .first();
      if (keep && keep.id > 0) {
        // remove what is not the first entry.
        await knex('medias')
          .where('filepath', duplicate.filepath)
          .whereNot('id', keep.id).del();
      }
    }
  }
}
RackNerd Billboard Banner
Share node:
/// .RELATED.DATA_STREAMSShared tags

Related posts

Scanning_database_for_correlations…

/// .UPLINK.SUBSCRIBE

Keep your deck synced with every new transmission from ATLASSC.NET. Subscribe over RSS or Atom, or wire the whole archive straight into your AI assistant with the MCP server.