Skip to main content
Legend
June 29, 2021
Question

Script snippet - removing duplicate terms in a list

  • June 29, 2021
  • 2 replies
  • 261 views

Short snippet that will remove duplicate terms in a list. I futzed around with this for a while trying to figure out the proper regex and methods to accomplish this.

I start with a list of terms, copy the first to a new array, globally delete, copy the next, and keep going until I'm through all of them. Thought someone might find this useful.

 

try{
var term = ''; //new string
var terma = []; //new array
term = '444\n111\n333\n444a\n111a\n333\n111\n333a\n333\n222\n444'; //sample string
term = term.replace(/\n\n/g, '\n'); //strip blank lines
term = term.replace(/\n$/, ''); //strip trailing return

term = term.replace(/^\n/, ''); //strip leading return
terma = term.split('\n'); //place into array
var clean = []; //new array
var n = 0; //counter
while(terma.length > 0 && terma[0] != ''){ //iterate through all items
clean[n] = terma[0]; //copy first item
var regterm = terma[0]; //term to replace
var regex = new RegExp('^' + regterm + '$','gm'); //build regex
term = terma.join('\n'); //makew string
term = term.replace(regex, ''); //remove exact term
term = term.replace(/\n\n/g, '\n'); //strip blank lines
term = term.replace(/^\n/, ''); //strip leading return
terma = term.split('\n'); //place into array
n = n + 1; //count
}
alert(clean);
}
catch(e){
alert (e + e.line);
}

This topic has been closed for replies.

2 replies

Kukurykus
Legend
June 30, 2021

 

strng = '444\n111\n333\n444a\n111a\n333\n111\n333a\n333\n222\n444'; arr = Array()
for(i in eval('({' + strng.replace(/(?:^|\n)(.*)?/g, '"$1": "", ') + '})')) arr.push(i); arr

 

Legend
June 30, 2021

Oh I'll have to look at this closer. I figured there was an easier way.

Kukurykus
Legend
June 30, 2021

 

arr1 = '''444\n111\n333\n444a\n111a\n333\n111\n333a
333\n222\n444'''.split('\n').sort(), arr2 = Array()
while(arr2.push(shft = arr1.shift()), arr1.length)
	arr1 = String(arr1.join(',').split
	(RegExp(shft + '(?:,|$)')))
	.match(/[^,]+/g); arr2

 

Stephen Marsh
Community Expert
Community Expert
June 29, 2021

Thank you for sharing!