I need a hint on how to deleted duplicate items in a array

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

I need a hint on how to deleted duplicate items in a array

Fernando Cabral
Hi

I have a sorted array that may contain several repeated items scattered all
over.

I have to do two different things at different times:
a) Eliminate the duplicates leaving a single specimen from each repeated
item;
b) Eliminate the duplicates but having a count of the original number.

So, if I have, say

A
B
B
C
D
D

In the first option, I want to have
A
B
C
D
In the second option, I want to have
1 A
2 B
1 C
2 D

Any hints on how to do this using some Gambas buit in method?

Note; Presently I have been doing it using external calls to
the utilities sort and uniq.

Regards

- fernando


--
Fernando Cabral
Blogue: http://fernandocabral.org
Twitter: http://twitter.com/fjcabral
e-mail: [hidden email]
Facebook: [hidden email]
Telegram: +55 (37) 99988-8868
Wickr ID: fernandocabral
WhatsApp: +55 (37) 99988-8868
Skype:  fernandojosecabral
Telefone fixo: +55 (37) 3521-2183
Telefone celular: +55 (37) 99988-8868

Enquanto houver no mundo uma só pessoa sem casa ou sem alimentos,
nenhum político ou cientista poderá se gabar de nada.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gambas-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gambas-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: I need a hint on how to deleted duplicate items in a array

Hans Lehmann
Hello,

look here:

8<-----------------------------------------------------------------------------------------
Public Function RemoveMultiple(aStringListe As String[]) As String[]
   Dim iCount As Integer
   Dim iIndex As Integer
   Dim sElement As String

   iIndex = 0 ' Initialisierung NICHT notwendig
   While iIndex < aStringListe.Count
     iCount = 0
     sElement = aStringListe[iIndex]
     While aStringListe.Find(sElement) <> -1
       Inc iCount
       aStringListe.Remove(aStringListe.Find(sElement))
     Wend
     If iCount Mod 2 = 1 Then
        aStringListe.Add(sElement, iIndex)
        Inc iIndex
     Endif ' iCount Mod 2 = 1 ?
   Wend

   Return aStringListe

End ' RemoveMultiple(...)
8<-----------------------------------------------------------------------------------------

Hans
gambas-buch.de

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gambas-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gambas-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: I need a hint on how to deleted duplicate items in a array

Tobias Boege-2
In reply to this post by Fernando Cabral
On Tue, 27 Jun 2017, Fernando Cabral wrote:

> Hi
>
> I have a sorted array that may contain several repeated items scattered all
> over.
>
> I have to do two different things at different times:
> a) Eliminate the duplicates leaving a single specimen from each repeated
> item;
> b) Eliminate the duplicates but having a count of the original number.
>
> So, if I have, say
>
> A
> B
> B
> C
> D
> D
>
> In the first option, I want to have
> A
> B
> C
> D
> In the second option, I want to have
> 1 A
> 2 B
> 1 C
> 2 D
>
> Any hints on how to do this using some Gambas buit in method?
>
> Note; Presently I have been doing it using external calls to
> the utilities sort and uniq.
>

Your first sentence is a bit confusing. First you say that your array is
sorted but then you say that duplicates may be scattered across the array.
There are notions of order (namely *preorder*) which are so weak that this
could happen, but are you actually dealing with a preorder on your items?
What are your items, anyway?

When I hear "sorted", I think of a partial order and if you have a partial
order, then sorted implies that duplicates are consecutive! Anyway, I don't
want to bore you with elementary concepts of order theory. There are ways
to handle preorders, partial orders and every stronger notion of order,
of course, from within Gambas. You simply have to ask a better question,
by giving more details.

If you have a sorting where duplicates are consecutive, the solution is
very easy: just go through the array linearly and kick out these consecutive
duplicates (which is precisely what uniq does), e.g. for integers:

  Dim aInts As Integer[] = ...
  Dim iInd, iLast As Integer

  If Not aInts.Count Then Return
  iLast = aInts[0]
  iInd = 1
  While iInd < aInts.Count
    If aInts[iInd] = iLast Then ' consecutive duplicate
      aInts.Remove(iInd, 1)
    Else
      iLast = aInts[iInd]
      Inc iInd
    Endif
  Wend

Note that the way I wrote it to get the idea across is not a linear-time
operation (it depends on the complexity of aInts.Remove()), but you can
achieve linear performance by writing better code. Think of it as an
exercise. (Of course, you can't hope to be more efficient than linear
time in a general situation.)

The counting task is solved with a similar pattern, but while you kick
an element out, you also increment a dedicated counter:

  Dim aInts As Integer[] = ...
  Dim aDups As New Integer[]
  Dim iInd, iLast As Integer

  If Not aInts.Count Then Return
  iLast = aInts[0]
  iInd = 1
  aDups.Add(0)
  While iInd < aInts.Count
    If aInts[iInd] = iLast Then ' consecutive duplicate
      aInts.Remove(iInd, 1)
      Inc aDups[aDups.Max]
    Else
      iLast = aInts[iInd]
      aDups.Add(0)
      Inc iInd
    Endif
  Wend

After this executed, the array aInts will not contain duplicates (supposing
it was sorted before) and aDups[i] will contain the number of duplicates of
the item aInts[i] that were removed.

Regards,
Tobi

--
"There's an old saying: Don't change anything... ever!" -- Mr. Monk

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gambas-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gambas-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: I need a hint on how to deleted duplicate items in a array

nando_f
In reply to this post by Hans Lehmann
Another very effective and simple would be:

You have your array with data
You create a new empty array.

Loop through each item in your array with data
If it's not in the new array, then add it.

Destroy the original array.
Keep the new one.
...something like (syntax may not be correct)

Public Function RemoveMultiple(a As String[]) As String[]

  Dim x as Integer
  Dim z as NEW STRING[]

  For x = 1 to a.count()
    if z.Find(a) = 0 Then z.Add(a[x])
  Next

  Return z

END

-Nando (Canada)




--
Open WebMail Project (http://openwebmail.org)


---------- Original Message -----------
From: Hans Lehmann <[hidden email]>
To: [hidden email]
Sent: Tue, 27 Jun 2017 15:51:19 +0200
Subject: Re: [Gambas-user] I need a hint on how to deleted duplicate items in a array

> Hello,
>
> look here:
>
> 8<-------------------------------------------------------------------------------
> ---------- Public Function RemoveMultiple(aStringListe As String[]) As String[]  
>  Dim iCount As Integer   Dim iIndex As Integer   Dim sElement As String
>
>    iIndex = 0 ' Initialisierung NICHT notwendig
>    While iIndex < aStringListe.Count
>      iCount = 0
>      sElement = aStringListe[iIndex]
>      While aStringListe.Find(sElement) <> -1
>        Inc iCount
>        aStringListe.Remove(aStringListe.Find(sElement))
>      Wend
>      If iCount Mod 2 = 1 Then
>         aStringListe.Add(sElement, iIndex)
>         Inc iIndex
>      Endif ' iCount Mod 2 = 1 ?
>    Wend
>
>    Return aStringListe
>
> End ' RemoveMultiple(...)
> 8<-------------------------------------------------------------------------------
> ----------
>
> Hans
> gambas-buch.de
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Gambas-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gambas-user
------- End of Original Message -------


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gambas-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gambas-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: I need a hint on how to deleted duplicate items in a array

Gianluigi
My two cents.

Public Sub Main()

  Dim sSort As String[] = ["A", "B", "B", "B", "C", "D", "D", "E", "E",
"E", "E", "F"]
  Dim sSame As String[] = sSort
  Dim bb As New Byte[]
  Dim sSingle As New String[]
  Dim i, n As Integer

  For i = 0 To sSort.Max
    If i < sSort.Max Then
      If sSort[i] = sSame[i + 1] Then
        Inc n
      Else
        sSingle.Push(sSort[i])
        bb.Push(n + 1)
        n = 0
      Endif
    Endif
  Next
  sSingle.Push(sSort[sSort.Max])
  bb.Push(n + 1)
  For i = 0 To sSingle.Max
    Print sSingle[i]
  Next
  For i = 0 To bb.Max
    Print bb[i] & sSingle[i]
  Next

End

Regards
Gianluigi

2017-06-27 16:33 GMT+02:00 <[hidden email]>:

> Another very effective and simple would be:
>
> You have your array with data
> You create a new empty array.
>
> Loop through each item in your array with data
> If it's not in the new array, then add it.
>
> Destroy the original array.
> Keep the new one.
> ...something like (syntax may not be correct)
>
> Public Function RemoveMultiple(a As String[]) As String[]
>
>   Dim x as Integer
>   Dim z as NEW STRING[]
>
>   For x = 1 to a.count()
>     if z.Find(a) = 0 Then z.Add(a[x])
>   Next
>
>   Return z
>
> END
>
> -Nando (Canada)
>
>
>
>
> --
> Open WebMail Project (http://openwebmail.org)
>
>
> ---------- Original Message -----------
> From: Hans Lehmann <[hidden email]>
> To: [hidden email]
> Sent: Tue, 27 Jun 2017 15:51:19 +0200
> Subject: Re: [Gambas-user] I need a hint on how to deleted duplicate items
> in a array
>
> > Hello,
> >
> > look here:
> >
> > 8<----------------------------------------------------------
> ---------------------
> > ---------- Public Function RemoveMultiple(aStringListe As String[]) As
> String[]
> >  Dim iCount As Integer   Dim iIndex As Integer   Dim sElement As String
> >
> >    iIndex = 0 ' Initialisierung NICHT notwendig
> >    While iIndex < aStringListe.Count
> >      iCount = 0
> >      sElement = aStringListe[iIndex]
> >      While aStringListe.Find(sElement) <> -1
> >        Inc iCount
> >        aStringListe.Remove(aStringListe.Find(sElement))
> >      Wend
> >      If iCount Mod 2 = 1 Then
> >         aStringListe.Add(sElement, iIndex)
> >         Inc iIndex
> >      Endif ' iCount Mod 2 = 1 ?
> >    Wend
> >
> >    Return aStringListe
> >
> > End ' RemoveMultiple(...)
> > 8<----------------------------------------------------------
> ---------------------
> > ----------
> >
> > Hans
> > gambas-buch.de
> >
> > ------------------------------------------------------------
> ------------------
> > Check out the vibrant tech community on one of the world's most
> > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > _______________________________________________
> > Gambas-user mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/gambas-user
> ------- End of Original Message -------
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Gambas-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gambas-user
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gambas-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gambas-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: I need a hint on how to deleted duplicate items in a array

nando_f
Well, there is complicated, then there is simplicity:
I tested this. Works for sorted, unsorted.
Can't be any simpler.

Public Function RemoveMultiple(a As String[]) As String[]

Dim x as Integer
Dim z as NEW STRING[]

For x = 1 to a.count()
  if z.Find(a) = 0 Then z.Add(a[x])
Next

'if you want it sorted, do it here
Return z

END

' - - - - -
use it this way:

myArray = RemoveMultiple(myArray)
  'the z array is now myArray.
  'the original array is destroyed because there are no references.



--
Open WebMail Project (http://openwebmail.org)


---------- Original Message -----------
From: Gianluigi <[hidden email]>
To: mailing list for gambas users <[hidden email]>
Sent: Tue, 27 Jun 2017 16:52:48 +0200
Subject: Re: [Gambas-user] I need a hint on how to deleted duplicate items in a array

> My two cents.
>
> Public Sub Main()
>
>   Dim sSort As String[] = ["A", "B", "B", "B", "C", "D", "D", "E", "E",
> "E", "E", "F"]
>   Dim sSame As String[] = sSort
>   Dim bb As New Byte[]
>   Dim sSingle As New String[]
>   Dim i, n As Integer
>
>   For i = 0 To sSort.Max
>     If i < sSort.Max Then
>       If sSort[i] = sSame[i + 1] Then
>         Inc n
>       Else
>         sSingle.Push(sSort[i])
>         bb.Push(n + 1)
>         n = 0
>       Endif
>     Endif
>   Next
>   sSingle.Push(sSort[sSort.Max])
>   bb.Push(n + 1)
>   For i = 0 To sSingle.Max
>     Print sSingle[i]
>   Next
>   For i = 0 To bb.Max
>     Print bb[i] & sSingle[i]
>   Next
>
> End
>
> Regards
> Gianluigi
>
> 2017-06-27 16:33 GMT+02:00 <[hidden email]>:
>
> > Another very effective and simple would be:
> >
> > You have your array with data
> > You create a new empty array.
> >
> > Loop through each item in your array with data
> > If it's not in the new array, then add it.
> >
> > Destroy the original array.
> > Keep the new one.
> > ...something like (syntax may not be correct)
> >
> > Public Function RemoveMultiple(a As String[]) As String[]
> >
> >   Dim x as Integer
> >   Dim z as NEW STRING[]
> >
> >   For x = 1 to a.count()
> >     if z.Find(a) = 0 Then z.Add(a[x])
> >   Next
> >
> >   Return z
> >
> > END
> >
> > -Nando (Canada)
> >
> >
> >
> >
> > --
> > Open WebMail Project (http://openwebmail.org)
> >
> >
> > ---------- Original Message -----------
> > From: Hans Lehmann <[hidden email]>
> > To: [hidden email]
> > Sent: Tue, 27 Jun 2017 15:51:19 +0200
> > Subject: Re: [Gambas-user] I need a hint on how to deleted duplicate items
> > in a array
> >
> > > Hello,
> > >
> > > look here:
> > >
> > > 8<----------------------------------------------------------
> > ---------------------
> > > ---------- Public Function RemoveMultiple(aStringListe As String[]) As
> > String[]
> > >  Dim iCount As Integer   Dim iIndex As Integer   Dim sElement As String
> > >
> > >    iIndex = 0 ' Initialisierung NICHT notwendig
> > >    While iIndex < aStringListe.Count
> > >      iCount = 0
> > >      sElement = aStringListe[iIndex]
> > >      While aStringListe.Find(sElement) <> -1
> > >        Inc iCount
> > >        aStringListe.Remove(aStringListe.Find(sElement))
> > >      Wend
> > >      If iCount Mod 2 = 1 Then
> > >         aStringListe.Add(sElement, iIndex)
> > >         Inc iIndex
> > >      Endif ' iCount Mod 2 = 1 ?
> > >    Wend
> > >
> > >    Return aStringListe
> > >
> > > End ' RemoveMultiple(...)
> > > 8<----------------------------------------------------------
> > ---------------------
> > > ----------
> > >
> > > Hans
> > > gambas-buch.de
> > >
> > > ------------------------------------------------------------
> > ------------------
> > > Check out the vibrant tech community on one of the world's most
> > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > > _______________________________________________
> > > Gambas-user mailing list
> > > [hidden email]
> > > https://lists.sourceforge.net/lists/listinfo/gambas-user
> > ------- End of Original Message -------
> >
> >
> > ------------------------------------------------------------
> > ------------------
> > Check out the vibrant tech community on one of the world's most
> > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > _______________________________________________
> > Gambas-user mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/gambas-user
> >
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Gambas-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gambas-user
------- End of Original Message -------


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gambas-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gambas-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: I need a hint on how to deleted duplicate items in a array

Fernando Cabral
In reply to this post by Tobias Boege-2
2017-06-27 11:29 GMT-03:00 Tobias Boege <[hidden email]>:

>
> Your first sentence is a bit confusing. First you say that your array is
> sorted but then you say that duplicates may be scattered across the array.
>
You are right. My fault. The array is sorted. What I meant by scattered was
that
pairs, duples, triplets or a bunch of duplicates may appear all over
interspersed with non-duplicated items.

My items are either words or sentences (extracted from an ODT file.
After the extraction, the words (or sentences) are sorted with the method
Array.sort(gb.descent).

After sorting it is much more efficient to search for the duplicates. And
it can be done
with some simple code (as some people have exemplified in this thread).

So, my question is basically if Gambas has some built in method do
eliminate duplicates.
The reason I am asking this is because I am new to Gambas, so I have found
myself coding
things that were not needed. For instance, I coded some functions to do
quicksort and bubble sort and then I found Array.sort () was available.
Therefore, I waisted my time coding those quicksort and bubble sort
functions.... :-(

Regards

- fernando


> If you have a sorting where duplicates are consecutive, the solution is
> very easy: just go through the array linearly and kick out these
> consecutive
> duplicates (which is precisely what uniq does), e.g. for integers:
>
>   Dim aInts As Integer[] = ...
>   Dim iInd, iLast As Integer
>
>   If Not aInts.Count Then Return
>   iLast = aInts[0]
>   iInd = 1
>   While iInd < aInts.Count
>     If aInts[iInd] = iLast Then ' consecutive duplicate
>       aInts.Remove(iInd, 1)
>     Else
>       iLast = aInts[iInd]
>       Inc iInd
>     Endif
>   Wend
>
> Note that the way I wrote it to get the idea across is not a linear-time
> operation (it depends on the complexity of aInts.Remove()), but you can
> achieve linear performance by writing better code. Think of it as an
> exercise. (Of course, you can't hope to be more efficient than linear
> time in a general situation.)
>
> The counting task is solved with a similar pattern, but while you kick
> an element out, you also increment a dedicated counter:
>
>   Dim aInts As Integer[] = ...
>   Dim aDups As New Integer[]
>   Dim iInd, iLast As Integer
>
>   If Not aInts.Count Then Return
>   iLast = aInts[0]
>   iInd = 1
>   aDups.Add(0)
>   While iInd < aInts.Count
>     If aInts[iInd] = iLast Then ' consecutive duplicate
>       aInts.Remove(iInd, 1)
>       Inc aDups[aDups.Max]
>     Else
>       iLast = aInts[iInd]
>       aDups.Add(0)
>       Inc iInd
>     Endif
>   Wend
>
> After this executed, the array aInts will not contain duplicates (supposing
> it was sorted before) and aDups[i] will contain the number of duplicates of
> the item aInts[i] that were removed.
>
> Regards,
> Tobi
>
> --
> "There's an old saying: Don't change anything... ever!" -- Mr. Monk
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Gambas-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gambas-user
>



--
Fernando Cabral
Blogue: http://fernandocabral.org
Twitter: http://twitter.com/fjcabral
e-mail: [hidden email]
Facebook: [hidden email]
Telegram: +55 (37) 99988-8868
Wickr ID: fernandocabral
WhatsApp: +55 (37) 99988-8868
Skype:  fernandojosecabral
Telefone fixo: +55 (37) 3521-2183
Telefone celular: +55 (37) 99988-8868

Enquanto houver no mundo uma só pessoa sem casa ou sem alimentos,
nenhum político ou cientista poderá se gabar de nada.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gambas-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gambas-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: I need a hint on how to deleted duplicate items in a array

Tobias Boege-2
On Tue, 27 Jun 2017, Fernando Cabral wrote:
> So, my question is basically if Gambas has some built in method do
> eliminate duplicates.
> The reason I am asking this is because I am new to Gambas, so I have found
> myself coding
> things that were not needed. For instance, I coded some functions to do
> quicksort and bubble sort and then I found Array.sort () was available.
> Therefore, I waisted my time coding those quicksort and bubble sort
> functions.... :-(
>

Ah, ok. I'm almost sure there is no built-in "uniq" function which gets
rid of consecutive duplicates, so you can go ahead and write your own :-)

--
"There's an old saying: Don't change anything... ever!" -- Mr. Monk

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gambas-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gambas-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: I need a hint on how to deleted duplicate items in a array

Fernando Cabral
In reply to this post by nando_f
Nando

The problem with this search and destroy method without pre-sorting is the
exponentional
growth in time needed to do the job.  If my math is not wrong, this is how
quickly it gets unmanageable:

Items / Comparisons needed (worst case scenario)
10 = 45
100 = 4,950
1000 = 499,500
1000 = 49,995,000

My program has to face a few thousand items, so not sorting does not seem a
good option.

Regards

- fernando



2017-06-27 12:59 GMT-03:00 <[hidden email]>:

> Well, there is complicated, then there is simplicity:
> I tested this. Works for sorted, unsorted.
> Can't be any simpler.
>
> Public Function RemoveMultiple(a As String[]) As String[]
>
> Dim x as Integer
> Dim z as NEW STRING[]
>
> For x = 1 to a.count()
>   if z.Find(a) = 0 Then z.Add(a[x])
> Next
>
> 'if you want it sorted, do it here
> Return z
>
> END
>
> ' - - - - -
> use it this way:
>
> myArray = RemoveMultiple(myArray)
>   'the z array is now myArray.
>   'the original array is destroyed because there are no references.
>
>
>
> --
> Open WebMail Project (http://openwebmail.org)
>
>
> ---------- Original Message -----------
> From: Gianluigi <[hidden email]>
> To: mailing list for gambas users <[hidden email]>
> Sent: Tue, 27 Jun 2017 16:52:48 +0200
> Subject: Re: [Gambas-user] I need a hint on how to deleted duplicate items
> in a array
>
> > My two cents.
> >
> > Public Sub Main()
> >
> >   Dim sSort As String[] = ["A", "B", "B", "B", "C", "D", "D", "E", "E",
> > "E", "E", "F"]
> >   Dim sSame As String[] = sSort
> >   Dim bb As New Byte[]
> >   Dim sSingle As New String[]
> >   Dim i, n As Integer
> >
> >   For i = 0 To sSort.Max
> >     If i < sSort.Max Then
> >       If sSort[i] = sSame[i + 1] Then
> >         Inc n
> >       Else
> >         sSingle.Push(sSort[i])
> >         bb.Push(n + 1)
> >         n = 0
> >       Endif
> >     Endif
> >   Next
> >   sSingle.Push(sSort[sSort.Max])
> >   bb.Push(n + 1)
> >   For i = 0 To sSingle.Max
> >     Print sSingle[i]
> >   Next
> >   For i = 0 To bb.Max
> >     Print bb[i] & sSingle[i]
> >   Next
> >
> > End
> >
> > Regards
> > Gianluigi
> >
> > 2017-06-27 16:33 GMT+02:00 <[hidden email]>:
> >
> > > Another very effective and simple would be:
> > >
> > > You have your array with data
> > > You create a new empty array.
> > >
> > > Loop through each item in your array with data
> > > If it's not in the new array, then add it.
> > >
> > > Destroy the original array.
> > > Keep the new one.
> > > ...something like (syntax may not be correct)
> > >
> > > Public Function RemoveMultiple(a As String[]) As String[]
> > >
> > >   Dim x as Integer
> > >   Dim z as NEW STRING[]
> > >
> > >   For x = 1 to a.count()
> > >     if z.Find(a) = 0 Then z.Add(a[x])
> > >   Next
> > >
> > >   Return z
> > >
> > > END
> > >
> > > -Nando (Canada)
> > >
> > >
> > >
> > >
> > > --
> > > Open WebMail Project (http://openwebmail.org)
> > >
> > >
> > > ---------- Original Message -----------
> > > From: Hans Lehmann <[hidden email]>
> > > To: [hidden email]
> > > Sent: Tue, 27 Jun 2017 15:51:19 +0200
> > > Subject: Re: [Gambas-user] I need a hint on how to deleted duplicate
> items
> > > in a array
> > >
> > > > Hello,
> > > >
> > > > look here:
> > > >
> > > > 8<----------------------------------------------------------
> > > ---------------------
> > > > ---------- Public Function RemoveMultiple(aStringListe As String[])
> As
> > > String[]
> > > >  Dim iCount As Integer   Dim iIndex As Integer   Dim sElement As
> String
> > > >
> > > >    iIndex = 0 ' Initialisierung NICHT notwendig
> > > >    While iIndex < aStringListe.Count
> > > >      iCount = 0
> > > >      sElement = aStringListe[iIndex]
> > > >      While aStringListe.Find(sElement) <> -1
> > > >        Inc iCount
> > > >        aStringListe.Remove(aStringListe.Find(sElement))
> > > >      Wend
> > > >      If iCount Mod 2 = 1 Then
> > > >         aStringListe.Add(sElement, iIndex)
> > > >         Inc iIndex
> > > >      Endif ' iCount Mod 2 = 1 ?
> > > >    Wend
> > > >
> > > >    Return aStringListe
> > > >
> > > > End ' RemoveMultiple(...)
> > > > 8<----------------------------------------------------------
> > > ---------------------
> > > > ----------
> > > >
> > > > Hans
> > > > gambas-buch.de
> > > >
> > > > ------------------------------------------------------------
> > > ------------------
> > > > Check out the vibrant tech community on one of the world's most
> > > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > > > _______________________________________________
> > > > Gambas-user mailing list
> > > > [hidden email]
> > > > https://lists.sourceforge.net/lists/listinfo/gambas-user
> > > ------- End of Original Message -------
> > >
> > >
> > > ------------------------------------------------------------
> > > ------------------
> > > Check out the vibrant tech community on one of the world's most
> > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > > _______________________________________________
> > > Gambas-user mailing list
> > > [hidden email]
> > > https://lists.sourceforge.net/lists/listinfo/gambas-user
> > >
> > ------------------------------------------------------------
> ------------------
> > Check out the vibrant tech community on one of the world's most
> > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > _______________________________________________
> > Gambas-user mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/gambas-user
> ------- End of Original Message -------
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Gambas-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gambas-user
>



--
Fernando Cabral
Blogue: http://fernandocabral.org
Twitter: http://twitter.com/fjcabral
e-mail: [hidden email]
Facebook: [hidden email]
Telegram: +55 (37) 99988-8868
Wickr ID: fernandocabral
WhatsApp: +55 (37) 99988-8868
Skype:  fernandojosecabral
Telefone fixo: +55 (37) 3521-2183
Telefone celular: +55 (37) 99988-8868

Enquanto houver no mundo uma só pessoa sem casa ou sem alimentos,
nenhum político ou cientista poderá se gabar de nada.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gambas-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gambas-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: I need a hint on how to deleted duplicate items in a array

Jussi Lahtinen
In reply to this post by nando_f
As Fernando stated your code is good only for small arrays. But if someone
is going to use it, here is correct implementation:

For x = 0 to a.Max
  if z.Find(a[x]) = -1 Then z.Add(a[x])
Next


z.Exist() might be faster... I don't know.



Jussi



On Tue, Jun 27, 2017 at 6:59 PM, <[hidden email]> wrote:

> Well, there is complicated, then there is simplicity:
> I tested this. Works for sorted, unsorted.
> Can't be any simpler.
>
> Public Function RemoveMultiple(a As String[]) As String[]
>
> Dim x as Integer
> Dim z as NEW STRING[]
>
> For x = 1 to a.count()
>   if z.Find(a) = 0 Then z.Add(a[x])
> Next
>
> 'if you want it sorted, do it here
> Return z
>
> END
>
> ' - - - - -
> use it this way:
>
> myArray = RemoveMultiple(myArray)
>   'the z array is now myArray.
>   'the original array is destroyed because there are no references.
>
>
>
> --
> Open WebMail Project (http://openwebmail.org)
>
>
> ---------- Original Message -----------
> From: Gianluigi <[hidden email]>
> To: mailing list for gambas users <[hidden email]>
> Sent: Tue, 27 Jun 2017 16:52:48 +0200
> Subject: Re: [Gambas-user] I need a hint on how to deleted duplicate items
> in a array
>
> > My two cents.
> >
> > Public Sub Main()
> >
> >   Dim sSort As String[] = ["A", "B", "B", "B", "C", "D", "D", "E", "E",
> > "E", "E", "F"]
> >   Dim sSame As String[] = sSort
> >   Dim bb As New Byte[]
> >   Dim sSingle As New String[]
> >   Dim i, n As Integer
> >
> >   For i = 0 To sSort.Max
> >     If i < sSort.Max Then
> >       If sSort[i] = sSame[i + 1] Then
> >         Inc n
> >       Else
> >         sSingle.Push(sSort[i])
> >         bb.Push(n + 1)
> >         n = 0
> >       Endif
> >     Endif
> >   Next
> >   sSingle.Push(sSort[sSort.Max])
> >   bb.Push(n + 1)
> >   For i = 0 To sSingle.Max
> >     Print sSingle[i]
> >   Next
> >   For i = 0 To bb.Max
> >     Print bb[i] & sSingle[i]
> >   Next
> >
> > End
> >
> > Regards
> > Gianluigi
> >
> > 2017-06-27 16:33 GMT+02:00 <[hidden email]>:
> >
> > > Another very effective and simple would be:
> > >
> > > You have your array with data
> > > You create a new empty array.
> > >
> > > Loop through each item in your array with data
> > > If it's not in the new array, then add it.
> > >
> > > Destroy the original array.
> > > Keep the new one.
> > > ...something like (syntax may not be correct)
> > >
> > > Public Function RemoveMultiple(a As String[]) As String[]
> > >
> > >   Dim x as Integer
> > >   Dim z as NEW STRING[]
> > >
> > >   For x = 1 to a.count()
> > >     if z.Find(a) = 0 Then z.Add(a[x])
> > >   Next
> > >
> > >   Return z
> > >
> > > END
> > >
> > > -Nando (Canada)
> > >
> > >
> > >
> > >
> > > --
> > > Open WebMail Project (http://openwebmail.org)
> > >
> > >
> > > ---------- Original Message -----------
> > > From: Hans Lehmann <[hidden email]>
> > > To: [hidden email]
> > > Sent: Tue, 27 Jun 2017 15:51:19 +0200
> > > Subject: Re: [Gambas-user] I need a hint on how to deleted duplicate
> items
> > > in a array
> > >
> > > > Hello,
> > > >
> > > > look here:
> > > >
> > > > 8<----------------------------------------------------------
> > > ---------------------
> > > > ---------- Public Function RemoveMultiple(aStringListe As String[])
> As
> > > String[]
> > > >  Dim iCount As Integer   Dim iIndex As Integer   Dim sElement As
> String
> > > >
> > > >    iIndex = 0 ' Initialisierung NICHT notwendig
> > > >    While iIndex < aStringListe.Count
> > > >      iCount = 0
> > > >      sElement = aStringListe[iIndex]
> > > >      While aStringListe.Find(sElement) <> -1
> > > >        Inc iCount
> > > >        aStringListe.Remove(aStringListe.Find(sElement))
> > > >      Wend
> > > >      If iCount Mod 2 = 1 Then
> > > >         aStringListe.Add(sElement, iIndex)
> > > >         Inc iIndex
> > > >      Endif ' iCount Mod 2 = 1 ?
> > > >    Wend
> > > >
> > > >    Return aStringListe
> > > >
> > > > End ' RemoveMultiple(...)
> > > > 8<----------------------------------------------------------
> > > ---------------------
> > > > ----------
> > > >
> > > > Hans
> > > > gambas-buch.de
> > > >
> > > > ------------------------------------------------------------
> > > ------------------
> > > > Check out the vibrant tech community on one of the world's most
> > > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > > > _______________________________________________
> > > > Gambas-user mailing list
> > > > [hidden email]
> > > > https://lists.sourceforge.net/lists/listinfo/gambas-user
> > > ------- End of Original Message -------
> > >
> > >
> > > ------------------------------------------------------------
> > > ------------------
> > > Check out the vibrant tech community on one of the world's most
> > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > > _______________________________________________
> > > Gambas-user mailing list
> > > [hidden email]
> > > https://lists.sourceforge.net/lists/listinfo/gambas-user
> > >
> > ------------------------------------------------------------
> ------------------
> > Check out the vibrant tech community on one of the world's most
> > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > _______________________________________________
> > Gambas-user mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/gambas-user
> ------- End of Original Message -------
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Gambas-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gambas-user
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gambas-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gambas-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: I need a hint on how to deleted duplicate items in a array

Fernando Cabral
Jussi said:

> As Fernando stated your code is good only for small arrays. But if someone
>is going to use it, here is correct implementation:

No, Jussi, I didn't say it is good only for small arrays. I said some
suggestions apply only
to small arrays because if I have to traverse the array again and again,
advancing one item at a time, and coming back to the next item, to repeat
it one more time, then time requirement will grow exponentially. This makes
most suggestion unusable for large arrays. The arrays I have might grow to
thousands and thousands os items.

Regards

- fernando






2017-06-27 15:43 GMT-03:00 Jussi Lahtinen <[hidden email]>:

> As Fernando stated your code is good only for small arrays. But if someone
> is going to use it, here is correct implementation:
>
> For x = 0 to a.Max
>   if z.Find(a[x]) = -1 Then z.Add(a[x])
> Next
>
>
> z.Exist() might be faster... I don't know.
>
>
>
> Jussi
>
>
>
> On Tue, Jun 27, 2017 at 6:59 PM, <[hidden email]> wrote:
>
> > Well, there is complicated, then there is simplicity:
> > I tested this. Works for sorted, unsorted.
> > Can't be any simpler.
> >
> > Public Function RemoveMultiple(a As String[]) As String[]
> >
> > Dim x as Integer
> > Dim z as NEW STRING[]
> >
> > For x = 1 to a.count()
> >   if z.Find(a) = 0 Then z.Add(a[x])
> > Next
> >
> > 'if you want it sorted, do it here
> > Return z
> >
> > END
> >
> > ' - - - - -
> > use it this way:
> >
> > myArray = RemoveMultiple(myArray)
> >   'the z array is now myArray.
> >   'the original array is destroyed because there are no references.
> >
> >
> >
> > --
> > Open WebMail Project (http://openwebmail.org)
> >
> >
> > ---------- Original Message -----------
> > From: Gianluigi <[hidden email]>
> > To: mailing list for gambas users <[hidden email]>
> > Sent: Tue, 27 Jun 2017 16:52:48 +0200
> > Subject: Re: [Gambas-user] I need a hint on how to deleted duplicate
> items
> > in a array
> >
> > > My two cents.
> > >
> > > Public Sub Main()
> > >
> > >   Dim sSort As String[] = ["A", "B", "B", "B", "C", "D", "D", "E", "E",
> > > "E", "E", "F"]
> > >   Dim sSame As String[] = sSort
> > >   Dim bb As New Byte[]
> > >   Dim sSingle As New String[]
> > >   Dim i, n As Integer
> > >
> > >   For i = 0 To sSort.Max
> > >     If i < sSort.Max Then
> > >       If sSort[i] = sSame[i + 1] Then
> > >         Inc n
> > >       Else
> > >         sSingle.Push(sSort[i])
> > >         bb.Push(n + 1)
> > >         n = 0
> > >       Endif
> > >     Endif
> > >   Next
> > >   sSingle.Push(sSort[sSort.Max])
> > >   bb.Push(n + 1)
> > >   For i = 0 To sSingle.Max
> > >     Print sSingle[i]
> > >   Next
> > >   For i = 0 To bb.Max
> > >     Print bb[i] & sSingle[i]
> > >   Next
> > >
> > > End
> > >
> > > Regards
> > > Gianluigi
> > >
> > > 2017-06-27 16:33 GMT+02:00 <[hidden email]>:
> > >
> > > > Another very effective and simple would be:
> > > >
> > > > You have your array with data
> > > > You create a new empty array.
> > > >
> > > > Loop through each item in your array with data
> > > > If it's not in the new array, then add it.
> > > >
> > > > Destroy the original array.
> > > > Keep the new one.
> > > > ...something like (syntax may not be correct)
> > > >
> > > > Public Function RemoveMultiple(a As String[]) As String[]
> > > >
> > > >   Dim x as Integer
> > > >   Dim z as NEW STRING[]
> > > >
> > > >   For x = 1 to a.count()
> > > >     if z.Find(a) = 0 Then z.Add(a[x])
> > > >   Next
> > > >
> > > >   Return z
> > > >
> > > > END
> > > >
> > > > -Nando (Canada)
> > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Open WebMail Project (http://openwebmail.org)
> > > >
> > > >
> > > > ---------- Original Message -----------
> > > > From: Hans Lehmann <[hidden email]>
> > > > To: [hidden email]
> > > > Sent: Tue, 27 Jun 2017 15:51:19 +0200
> > > > Subject: Re: [Gambas-user] I need a hint on how to deleted duplicate
> > items
> > > > in a array
> > > >
> > > > > Hello,
> > > > >
> > > > > look here:
> > > > >
> > > > > 8<----------------------------------------------------------
> > > > ---------------------
> > > > > ---------- Public Function RemoveMultiple(aStringListe As String[])
> > As
> > > > String[]
> > > > >  Dim iCount As Integer   Dim iIndex As Integer   Dim sElement As
> > String
> > > > >
> > > > >    iIndex = 0 ' Initialisierung NICHT notwendig
> > > > >    While iIndex < aStringListe.Count
> > > > >      iCount = 0
> > > > >      sElement = aStringListe[iIndex]
> > > > >      While aStringListe.Find(sElement) <> -1
> > > > >        Inc iCount
> > > > >        aStringListe.Remove(aStringListe.Find(sElement))
> > > > >      Wend
> > > > >      If iCount Mod 2 = 1 Then
> > > > >         aStringListe.Add(sElement, iIndex)
> > > > >         Inc iIndex
> > > > >      Endif ' iCount Mod 2 = 1 ?
> > > > >    Wend
> > > > >
> > > > >    Return aStringListe
> > > > >
> > > > > End ' RemoveMultiple(...)
> > > > > 8<----------------------------------------------------------
> > > > ---------------------
> > > > > ----------
> > > > >
> > > > > Hans
> > > > > gambas-buch.de
> > > > >
> > > > > ------------------------------------------------------------
> > > > ------------------
> > > > > Check out the vibrant tech community on one of the world's most
> > > > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > > > > _______________________________________________
> > > > > Gambas-user mailing list
> > > > > [hidden email]
> > > > > https://lists.sourceforge.net/lists/listinfo/gambas-user
> > > > ------- End of Original Message -------
> > > >
> > > >
> > > > ------------------------------------------------------------
> > > > ------------------
> > > > Check out the vibrant tech community on one of the world's most
> > > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > > > _______________________________________________
> > > > Gambas-user mailing list
> > > > [hidden email]
> > > > https://lists.sourceforge.net/lists/listinfo/gambas-user
> > > >
> > > ------------------------------------------------------------
> > ------------------
> > > Check out the vibrant tech community on one of the world's most
> > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > > _______________________________________________
> > > Gambas-user mailing list
> > > [hidden email]
> > > https://lists.sourceforge.net/lists/listinfo/gambas-user
> > ------- End of Original Message -------
> >
> >
> > ------------------------------------------------------------
> > ------------------
> > Check out the vibrant tech community on one of the world's most
> > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > _______________________________________________
> > Gambas-user mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/gambas-user
> >
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Gambas-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gambas-user
>



--
Fernando Cabral
Blogue: http://fernandocabral.org
Twitter: http://twitter.com/fjcabral
e-mail: [hidden email]
Facebook: [hidden email]
Telegram: +55 (37) 99988-8868
Wickr ID: fernandocabral
WhatsApp: +55 (37) 99988-8868
Skype:  fernandojosecabral
Telefone fixo: +55 (37) 3521-2183
Telefone celular: +55 (37) 99988-8868

Enquanto houver no mundo uma só pessoa sem casa ou sem alimentos,
nenhum político ou cientista poderá se gabar de nada.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gambas-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gambas-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: I need a hint on how to deleted duplicate items in a array

Fabien Bodard-4
The best way is the nando one ... at least for gambas.

As you have not to matter about what is the index value or the order,
the walk ahead option is the better.


Then Fernando ... for big, big things... I think you need to use a DB.
Or a native language.... maybe a sqlite memory structure can be good.

2017-06-27 20:51 GMT+02:00 Fernando Cabral <[hidden email]>:

> Jussi said:
>
>> As Fernando stated your code is good only for small arrays. But if someone
>>is going to use it, here is correct implementation:
>
> No, Jussi, I didn't say it is good only for small arrays. I said some
> suggestions apply only
> to small arrays because if I have to traverse the array again and again,
> advancing one item at a time, and coming back to the next item, to repeat
> it one more time, then time requirement will grow exponentially. This makes
> most suggestion unusable for large arrays. The arrays I have might grow to
> thousands and thousands os items.
>
> Regards
>
> - fernando
>
>
>
>
>
>
> 2017-06-27 15:43 GMT-03:00 Jussi Lahtinen <[hidden email]>:
>
>> As Fernando stated your code is good only for small arrays. But if someone
>> is going to use it, here is correct implementation:
>>
>> For x = 0 to a.Max
>>   if z.Find(a[x]) = -1 Then z.Add(a[x])
>> Next
>>
>>
>> z.Exist() might be faster... I don't know.
>>
>>
>>
>> Jussi
>>
>>
>>
>> On Tue, Jun 27, 2017 at 6:59 PM, <[hidden email]> wrote:
>>
>> > Well, there is complicated, then there is simplicity:
>> > I tested this. Works for sorted, unsorted.
>> > Can't be any simpler.
>> >
>> > Public Function RemoveMultiple(a As String[]) As String[]
>> >
>> > Dim x as Integer
>> > Dim z as NEW STRING[]
>> >
>> > For x = 1 to a.count()
>> >   if z.Find(a) = 0 Then z.Add(a[x])
>> > Next
>> >
>> > 'if you want it sorted, do it here
>> > Return z
>> >
>> > END
>> >
>> > ' - - - - -
>> > use it this way:
>> >
>> > myArray = RemoveMultiple(myArray)
>> >   'the z array is now myArray.
>> >   'the original array is destroyed because there are no references.
>> >
>> >
>> >
>> > --
>> > Open WebMail Project (http://openwebmail.org)
>> >
>> >
>> > ---------- Original Message -----------
>> > From: Gianluigi <[hidden email]>
>> > To: mailing list for gambas users <[hidden email]>
>> > Sent: Tue, 27 Jun 2017 16:52:48 +0200
>> > Subject: Re: [Gambas-user] I need a hint on how to deleted duplicate
>> items
>> > in a array
>> >
>> > > My two cents.
>> > >
>> > > Public Sub Main()
>> > >
>> > >   Dim sSort As String[] = ["A", "B", "B", "B", "C", "D", "D", "E", "E",
>> > > "E", "E", "F"]
>> > >   Dim sSame As String[] = sSort
>> > >   Dim bb As New Byte[]
>> > >   Dim sSingle As New String[]
>> > >   Dim i, n As Integer
>> > >
>> > >   For i = 0 To sSort.Max
>> > >     If i < sSort.Max Then
>> > >       If sSort[i] = sSame[i + 1] Then
>> > >         Inc n
>> > >       Else
>> > >         sSingle.Push(sSort[i])
>> > >         bb.Push(n + 1)
>> > >         n = 0
>> > >       Endif
>> > >     Endif
>> > >   Next
>> > >   sSingle.Push(sSort[sSort.Max])
>> > >   bb.Push(n + 1)
>> > >   For i = 0 To sSingle.Max
>> > >     Print sSingle[i]
>> > >   Next
>> > >   For i = 0 To bb.Max
>> > >     Print bb[i] & sSingle[i]
>> > >   Next
>> > >
>> > > End
>> > >
>> > > Regards
>> > > Gianluigi
>> > >
>> > > 2017-06-27 16:33 GMT+02:00 <[hidden email]>:
>> > >
>> > > > Another very effective and simple would be:
>> > > >
>> > > > You have your array with data
>> > > > You create a new empty array.
>> > > >
>> > > > Loop through each item in your array with data
>> > > > If it's not in the new array, then add it.
>> > > >
>> > > > Destroy the original array.
>> > > > Keep the new one.
>> > > > ...something like (syntax may not be correct)
>> > > >
>> > > > Public Function RemoveMultiple(a As String[]) As String[]
>> > > >
>> > > >   Dim x as Integer
>> > > >   Dim z as NEW STRING[]
>> > > >
>> > > >   For x = 1 to a.count()
>> > > >     if z.Find(a) = 0 Then z.Add(a[x])
>> > > >   Next
>> > > >
>> > > >   Return z
>> > > >
>> > > > END
>> > > >
>> > > > -Nando (Canada)
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > Open WebMail Project (http://openwebmail.org)
>> > > >
>> > > >
>> > > > ---------- Original Message -----------
>> > > > From: Hans Lehmann <[hidden email]>
>> > > > To: [hidden email]
>> > > > Sent: Tue, 27 Jun 2017 15:51:19 +0200
>> > > > Subject: Re: [Gambas-user] I need a hint on how to deleted duplicate
>> > items
>> > > > in a array
>> > > >
>> > > > > Hello,
>> > > > >
>> > > > > look here:
>> > > > >
>> > > > > 8<----------------------------------------------------------
>> > > > ---------------------
>> > > > > ---------- Public Function RemoveMultiple(aStringListe As String[])
>> > As
>> > > > String[]
>> > > > >  Dim iCount As Integer   Dim iIndex As Integer   Dim sElement As
>> > String
>> > > > >
>> > > > >    iIndex = 0 ' Initialisierung NICHT notwendig
>> > > > >    While iIndex < aStringListe.Count
>> > > > >      iCount = 0
>> > > > >      sElement = aStringListe[iIndex]
>> > > > >      While aStringListe.Find(sElement) <> -1
>> > > > >        Inc iCount
>> > > > >        aStringListe.Remove(aStringListe.Find(sElement))
>> > > > >      Wend
>> > > > >      If iCount Mod 2 = 1 Then
>> > > > >         aStringListe.Add(sElement, iIndex)
>> > > > >         Inc iIndex
>> > > > >      Endif ' iCount Mod 2 = 1 ?
>> > > > >    Wend
>> > > > >
>> > > > >    Return aStringListe
>> > > > >
>> > > > > End ' RemoveMultiple(...)
>> > > > > 8<----------------------------------------------------------
>> > > > ---------------------
>> > > > > ----------
>> > > > >
>> > > > > Hans
>> > > > > gambas-buch.de
>> > > > >
>> > > > > ------------------------------------------------------------
>> > > > ------------------
>> > > > > Check out the vibrant tech community on one of the world's most
>> > > > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> > > > > _______________________________________________
>> > > > > Gambas-user mailing list
>> > > > > [hidden email]
>> > > > > https://lists.sourceforge.net/lists/listinfo/gambas-user
>> > > > ------- End of Original Message -------
>> > > >
>> > > >
>> > > > ------------------------------------------------------------
>> > > > ------------------
>> > > > Check out the vibrant tech community on one of the world's most
>> > > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> > > > _______________________________________________
>> > > > Gambas-user mailing list
>> > > > [hidden email]
>> > > > https://lists.sourceforge.net/lists/listinfo/gambas-user
>> > > >
>> > > ------------------------------------------------------------
>> > ------------------
>> > > Check out the vibrant tech community on one of the world's most
>> > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> > > _______________________________________________
>> > > Gambas-user mailing list
>> > > [hidden email]
>> > > https://lists.sourceforge.net/lists/listinfo/gambas-user
>> > ------- End of Original Message -------
>> >
>> >
>> > ------------------------------------------------------------
>> > ------------------
>> > Check out the vibrant tech community on one of the world's most
>> > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> > _______________________________________________
>> > Gambas-user mailing list
>> > [hidden email]
>> > https://lists.sourceforge.net/lists/listinfo/gambas-user
>> >
>> ------------------------------------------------------------
>> ------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> _______________________________________________
>> Gambas-user mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/gambas-user
>>
>
>
>
> --
> Fernando Cabral
> Blogue: http://fernandocabral.org
> Twitter: http://twitter.com/fjcabral
> e-mail: [hidden email]
> Facebook: [hidden email]
> Telegram: +55 (37) 99988-8868
> Wickr ID: fernandocabral
> WhatsApp: +55 (37) 99988-8868
> Skype:  fernandojosecabral
> Telefone fixo: +55 (37) 3521-2183
> Telefone celular: +55 (37) 99988-8868
>
> Enquanto houver no mundo uma só pessoa sem casa ou sem alimentos,
> nenhum político ou cientista poderá se gabar de nada.
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Gambas-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gambas-user



--
Fabien Bodard

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gambas-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gambas-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: I need a hint on how to deleted duplicate items in a array

Fernando Cabral
2017-06-30 7:44 GMT-03:00 Fabien Bodard <[hidden email]>:

> The best way is the nando one ... at least for gambas.
>
> As you have not to matter about what is the index value or the order,
> the walk ahead option is the better.
>
>
> Then Fernando ... for big, big things... I think you need to use a DB.
> Or a native language.... maybe a sqlite memory structure can be good.
>

Fabien, since this is a one-time only thing, I don't think I'd be better
off witha database.
Basically, I read a text file an then break it down into words, sentences
and paragraphs.
Next I  count the items in each array (words, sentences paragraphs).
Array.count works wonderfully.
After that, have to eliminate the duplicate words (Array.words). But in
doing it, al also have to count
how many times each word appeared.

Finally I sort the Array.Sentences and the Array.Paragraphs by size
(string.len()). The Array.WOrds are
sorted by count + lenght. This is all woring good.

So, my quest is for the fastest way do eliminate the words duplicates while
I count them.
For the time being, here is a working solution based on system' s sort |
uniq:

Here is one of the versions I have been using:

Exec ["/usr/bin/uniq", "Unsorted.txt", "Sorted.srt2"] Wait
Exec ["/usr/bin/uniq", "-ci", "SortedWords.srt2",  SortedWords.srt3"] Wait
Exec ["/usr/bin/sort", "-bnr", SortedWords.srt3] To UniqWords

WordArray = split (UniqWords, "\n")

So, I end up with the result I want. It's effective. Now, it would be more
elegant If I could do the same
with Gambas. Of course, the sorting would be easy with the builting
WordArray.sort ().
But how about te '"/usr/bin/uniq", "-ci" ...' part?

Regards

- fernando



> >
> >
> >
> >
> >
> > 2017-06-27 15:43 GMT-03:00 Jussi Lahtinen <[hidden email]>:
> >
> >> As Fernando stated your code is good only for small arrays. But if
> someone
> >> is going to use it, here is correct implementation:
> >>
> >> For x = 0 to a.Max
> >>   if z.Find(a[x]) = -1 Then z.Add(a[x])
> >> Next
> >>
> >>
> >> z.Exist() might be faster... I don't know.
> >>
> >>
> >>
> >> Jussi
> >>
> >>
> >>
> >> On Tue, Jun 27, 2017 at 6:59 PM, <[hidden email]> wrote:
> >>
> >> > Well, there is complicated, then there is simplicity:
> >> > I tested this. Works for sorted, unsorted.
> >> > Can't be any simpler.
> >> >
> >> > Public Function RemoveMultiple(a As String[]) As String[]
> >> >
> >> > Dim x as Integer
> >> > Dim z as NEW STRING[]
> >> >
> >> > For x = 1 to a.count()
> >> >   if z.Find(a) = 0 Then z.Add(a[x])
> >> > Next
> >> >
> >> > 'if you want it sorted, do it here
> >> > Return z
> >> >
> >> > END
> >> >
> >> > ' - - - - -
> >> > use it this way:
> >> >
> >> > myArray = RemoveMultiple(myArray)
> >> >   'the z array is now myArray.
> >> >   'the original array is destroyed because there are no references.
> >> >
> >> >
> >> >
> >> > --
> >> > Open WebMail Project (http://openwebmail.org)
> >> >
> >> >
> >> > ---------- Original Message -----------
> >> > From: Gianluigi <[hidden email]>
> >> > To: mailing list for gambas users <[hidden email]>
> >> > Sent: Tue, 27 Jun 2017 16:52:48 +0200
> >> > Subject: Re: [Gambas-user] I need a hint on how to deleted duplicate
> >> items
> >> > in a array
> >> >
> >> > > My two cents.
> >> > >
> >> > > Public Sub Main()
> >> > >
> >> > >   Dim sSort As String[] = ["A", "B", "B", "B", "C", "D", "D", "E",
> "E",
> >> > > "E", "E", "F"]
> >> > >   Dim sSame As String[] = sSort
> >> > >   Dim bb As New Byte[]
> >> > >   Dim sSingle As New String[]
> >> > >   Dim i, n As Integer
> >> > >
> >> > >   For i = 0 To sSort.Max
> >> > >     If i < sSort.Max Then
> >> > >       If sSort[i] = sSame[i + 1] Then
> >> > >         Inc n
> >> > >       Else
> >> > >         sSingle.Push(sSort[i])
> >> > >         bb.Push(n + 1)
> >> > >         n = 0
> >> > >       Endif
> >> > >     Endif
> >> > >   Next
> >> > >   sSingle.Push(sSort[sSort.Max])
> >> > >   bb.Push(n + 1)
> >> > >   For i = 0 To sSingle.Max
> >> > >     Print sSingle[i]
> >> > >   Next
> >> > >   For i = 0 To bb.Max
> >> > >     Print bb[i] & sSingle[i]
> >> > >   Next
> >> > >
> >> > > End
> >> > >
> >> > > Regards
> >> > > Gianluigi
> >> > >
> >> > > 2017-06-27 16:33 GMT+02:00 <[hidden email]>:
> >> > >
> >> > > > Another very effective and simple would be:
> >> > > >
> >> > > > You have your array with data
> >> > > > You create a new empty array.
> >> > > >
> >> > > > Loop through each item in your array with data
> >> > > > If it's not in the new array, then add it.
> >> > > >
> >> > > > Destroy the original array.
> >> > > > Keep the new one.
> >> > > > ...something like (syntax may not be correct)
> >> > > >
> >> > > > Public Function RemoveMultiple(a As String[]) As String[]
> >> > > >
> >> > > >   Dim x as Integer
> >> > > >   Dim z as NEW STRING[]
> >> > > >
> >> > > >   For x = 1 to a.count()
> >> > > >     if z.Find(a) = 0 Then z.Add(a[x])
> >> > > >   Next
> >> > > >
> >> > > >   Return z
> >> > > >
> >> > > > END
> >> > > >
> >> > > > -Nando (Canada)
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > Open WebMail Project (http://openwebmail.org)
> >> > > >
> >> > > >
> >> > > > ---------- Original Message -----------
> >> > > > From: Hans Lehmann <[hidden email]>
> >> > > > To: [hidden email]
> >> > > > Sent: Tue, 27 Jun 2017 15:51:19 +0200
> >> > > > Subject: Re: [Gambas-user] I need a hint on how to deleted
> duplicate
> >> > items
> >> > > > in a array
> >> > > >
> >> > > > > Hello,
> >> > > > >
> >> > > > > look here:
> >> > > > >
> >> > > > > 8<----------------------------------------------------------
> >> > > > ---------------------
> >> > > > > ---------- Public Function RemoveMultiple(aStringListe As
> String[])
> >> > As
> >> > > > String[]
> >> > > > >  Dim iCount As Integer   Dim iIndex As Integer   Dim sElement As
> >> > String
> >> > > > >
> >> > > > >    iIndex = 0 ' Initialisierung NICHT notwendig
> >> > > > >    While iIndex < aStringListe.Count
> >> > > > >      iCount = 0
> >> > > > >      sElement = aStringListe[iIndex]
> >> > > > >      While aStringListe.Find(sElement) <> -1
> >> > > > >        Inc iCount
> >> > > > >        aStringListe.Remove(aStringListe.Find(sElement))
> >> > > > >      Wend
> >> > > > >      If iCount Mod 2 = 1 Then
> >> > > > >         aStringListe.Add(sElement, iIndex)
> >> > > > >         Inc iIndex
> >> > > > >      Endif ' iCount Mod 2 = 1 ?
> >> > > > >    Wend
> >> > > > >
> >> > > > >    Return aStringListe
> >> > > > >
> >> > > > > End ' RemoveMultiple(...)
> >> > > > > 8<----------------------------------------------------------
> >> > > > ---------------------
> >> > > > > ----------
> >> > > > >
> >> > > > > Hans
> >> > > > > gambas-buch.de
> >> > > > >
> >> > > > > ------------------------------------------------------------
> >> > > > ------------------
> >> > > > > Check out the vibrant tech community on one of the world's most
> >> > > > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> >> > > > > _______________________________________________
> >> > > > > Gambas-user mailing list
> >> > > > > [hidden email]
> >> > > > > https://lists.sourceforge.net/lists/listinfo/gambas-user
> >> > > > ------- End of Original Message -------
> >> > > >
> >> > > >
> >> > > > ------------------------------------------------------------
> >> > > > ------------------
> >> > > > Check out the vibrant tech community on one of the world's most
> >> > > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> >> > > > _______________________________________________
> >> > > > Gambas-user mailing list
> >> > > > [hidden email]
> >> > > > https://lists.sourceforge.net/lists/listinfo/gambas-user
> >> > > >
> >> > > ------------------------------------------------------------
> >> > ------------------
> >> > > Check out the vibrant tech community on one of the world's most
> >> > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> >> > > _______________________________________________
> >> > > Gambas-user mailing list
> >> > > [hidden email]
> >> > > https://lists.sourceforge.net/lists/listinfo/gambas-user
> >> > ------- End of Original Message -------
> >> >
> >> >
> >> > ------------------------------------------------------------
> >> > ------------------
> >> > Check out the vibrant tech community on one of the world's most
> >> > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> >> > _______________________________________________
> >> > Gambas-user mailing list
> >> > [hidden email]
> >> > https://lists.sourceforge.net/lists/listinfo/gambas-user
> >> >
> >> ------------------------------------------------------------
> >> ------------------
> >> Check out the vibrant tech community on one of the world's most
> >> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> >> _______________________________________________
> >> Gambas-user mailing list
> >> [hidden email]
> >> https://lists.sourceforge.net/lists/listinfo/gambas-user
> >>
> >
> >
> >
> > --
> > Fernando Cabral
> > Blogue: http://fernandocabral.org
> > Twitter: http://twitter.com/fjcabral
> > e-mail: [hidden email]
> > Facebook: [hidden email]
> > Telegram: +55 (37) 99988-8868
> > Wickr ID: fernandocabral
> > WhatsApp: +55 (37) 99988-8868
> > Skype:  fernandojosecabral
> > Telefone fixo: +55 (37) 3521-2183
> > Telefone celular: +55 (37) 99988-8868
> >
> > Enquanto houver no mundo uma só pessoa sem casa ou sem alimentos,
> > nenhum político ou cientista poderá se gabar de nada.
> > ------------------------------------------------------------
> ------------------
> > Check out the vibrant tech community on one of the world's most
> > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > _______________________________________________
> > Gambas-user mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/gambas-user
>
>
>
> --
> Fabien Bodard
>



--
Fernando Cabral
Blogue: http://fernandocabral.org
Twitter: http://twitter.com/fjcabral
e-mail: [hidden email]
Facebook: [hidden email]
Telegram: +55 (37) 99988-8868
Wickr ID: fernandocabral
WhatsApp: +55 (37) 99988-8868
Skype:  fernandojosecabral
Telefone fixo: +55 (37) 3521-2183
Telefone celular: +55 (37) 99988-8868

Enquanto houver no mundo uma só pessoa sem casa ou sem alimentos,
nenhum político ou cientista poderá se gabar de nada.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gambas-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gambas-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: I need a hint on how to deleted duplicate items in a array

zxMarce
On 30/06/17 08:20, Fernando Cabral wrote:

> 2017-06-30 7:44 GMT-03:00 Fabien Bodard <[hidden email]>:
>> The best way is the nando one ... at least for gambas.
>> As you have not to matter about what is the index value or the order,
>> the walk ahead option is the better.
>> Then Fernando ... for big, big things... I think you need to use a DB.
>> Or a native language.... maybe a sqlite memory structure can be good.
> Fabien, since this is a one-time only thing, I don't think I'd be
> better off witha database.
> Basically, I read a text file an then break it down into words,
> sentences and paragraphs.
> Next I  count the items in each array (words, sentences paragraphs).
> Array.count works wonderfully.
> After that, have to eliminate the duplicate words (Array.words). But
> in doing it, al also have to count how many times each word appeared.
> Finally I sort the Array.Sentences and the Array.Paragraphs by size
> (string.len()). The Array.WOrds are sorted by count + lenght. This is
> all woring good.
> So, my quest is for the fastest way do eliminate the words duplicates
> while I count them.
> For the time being, here is a working solution based on system' s sort
> | uniq:
> Here is one of the versions I have been using:
> Exec ["/usr/bin/uniq", "Unsorted.txt", "Sorted.srt2"] Wait
> Exec ["/usr/bin/uniq", "-ci", "SortedWords.srt2",  SortedWords.srt3"] Wait
> Exec ["/usr/bin/sort", "-bnr", SortedWords.srt3] To UniqWords
> WordArray = split (UniqWords, "\n")
> So, I end up with the result I want. It's effective. Now, it would be
> more elegant If I could do the same with Gambas. Of course, the
> sorting would be easy with the builting WordArray.sort ().
> But how about te '"/usr/bin/uniq", "-ci" ...' part?
> Regards
> - fernando

Not tried, but for the duplicate count, what about iterating the word
array copying each word to a keyed collection?
For any new given word, the value (item) added would be 1 (integer), and
the key would be UCase(word$).
If an error happens, the handler would just Inc the keyed Item value. So
(please note my syntax may be slightly off, especially in If Error):

Public Function CountWordsInArray(sortedWordArray As String[]) As Collection

  Dim wordCount As New Collection
  Dim currentWord As String = Null

  For Each currentWord In sortedWordArray

    Try wordCount.Add(1, UCase$(currentWord))
    If Error Then
      Inc wordCount(UCase$(currentWord))
      Error.Clear 'Is this needed, or even correct?
    End If

  Next

  Return (wordCollection)

End

The returned collection should be sorted if the array was, and for each
item you will have a numeric count as the item and the word as the key.
Hope it helps,
zxMarce.


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gambas-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gambas-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: I need a hint on how to deleted duplicate items in a array

Tobias Boege-2
In reply to this post by Fernando Cabral
On Fri, 30 Jun 2017, Fernando Cabral wrote:

> 2017-06-30 7:44 GMT-03:00 Fabien Bodard <[hidden email]>:
>
> > The best way is the nando one ... at least for gambas.
> >
> > As you have not to matter about what is the index value or the order,
> > the walk ahead option is the better.
> >
> >
> > Then Fernando ... for big, big things... I think you need to use a DB.
> > Or a native language.... maybe a sqlite memory structure can be good.
> >
>
> Fabien, since this is a one-time only thing, I don't think I'd be better
> off witha database.
> Basically, I read a text file an then break it down into words, sentences
> and paragraphs.
> Next I  count the items in each array (words, sentences paragraphs).
> Array.count works wonderfully.
> After that, have to eliminate the duplicate words (Array.words). But in
> doing it, al also have to count
> how many times each word appeared.
>
> Finally I sort the Array.Sentences and the Array.Paragraphs by size
> (string.len()). The Array.WOrds are
> sorted by count + lenght. This is all woring good.
>
> So, my quest is for the fastest way do eliminate the words duplicates while
> I count them.
> For the time being, here is a working solution based on system' s sort |
> uniq:
>
> Here is one of the versions I have been using:
>
> Exec ["/usr/bin/uniq", "Unsorted.txt", "Sorted.srt2"] Wait
> Exec ["/usr/bin/uniq", "-ci", "SortedWords.srt2",  SortedWords.srt3"] Wait
> Exec ["/usr/bin/sort", "-bnr", SortedWords.srt3] To UniqWords
>
Are those temporary files? You can avoid those by piping your data into the
processes and reading their output directly. Otherwise the Temp$() function
gives you better temporary files.

> WordArray = split (UniqWords, "\n")
>
> So, I end up with the result I want. It's effective. Now, it would be more
> elegant If I could do the same
> with Gambas. Of course, the sorting would be easy with the builting
> WordArray.sort ().
> But how about te '"/usr/bin/uniq", "-ci" ...' part?
>

I feel like my other mail answered this, but I can give you another version
of that routine (which I said I would leave as an exercise to you):

  ' Remove duplicates in an array like "uniq -ci". String comparison is
  ' case insensitive. The i-th entry in the returned array counts how many
  ' times aStrings[i] (in the de-duplicated array) was present in the input.
  ' The data in ~aStrings~ is overridden. Assumes the array is sorted.
  Private Function Uniq(aStrings As String[]) As Integer[]
    Dim iSrc, iLast As Integer
    Dim aCount As New Integer[](aStrings.Count)

    If Not aStrings.Count Then Return []
    iLast = 0
    aCount[iLast] = 1
    For iSrc = 1 To aStrings.Max
      If String.Comp(aStrings[iSrc], aStrings[iLast], gb.IgnoreCase) Then
        Inc iLast
        aStrings[iLast] = aStrings[iSrc]
        aCount[iLast] = 1
      Else
        Inc aCount[iLast]
      Endif
    Next

    ' Now shrink the arrays to the memory they actually need
    aStrings.Resize(iLast + 1)
    aCount.Resize(iLast + 1)
    Return aCount
  End

What, in my opinion, is at least theoretically better here than the other
proposed solutions is that it runs in linear time, while nando's is
quadratic[*]. (Of course, if you sort beforehand, it will become n*log(n),
which is still better than quadratic.)

Attached is a test script with some words. It runs the sort + uniq utilities
first and then Array.Sort() + the Uniq() function above. The program then
prints the *diff* between the two outputs. I get an empty diff, meaning that
my Gambas routines produce exactly the same output as the shell utilities.

Regards,
Tobi

[*] He calls array functions Add() and Find() inside a For loop that runs
    over an array of size n. Adding elements to an array or searching an
    array have themselves worst-case linear complexity, giving quadratic
    overall. My implementation reserves some more space in advance to
    avoid calling Add() in a loop. Since the array is sorted, we can go
    without Find(), too. Actually, as you may know, adding an element to
    the end of an array can be implemented in amortized constant time
    (as C++'s std::vector does), by wasting space, but AFAICS Gambas
    doesn't do this, but I could be wrong.

--
"There's an old saying: Don't change anything... ever!" -- Mr. Monk

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gambas-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gambas-user

sortuniq.gbs3 (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: I need a hint on how to deleted duplicate items in a array

Gianluigi
What was wrong in my example which meant this?

Public Sub Main()

  Dim sSort As String[] = ["A", "B", "B", "B", "C", "D", "D", "E", "E",
"E", "E", "F"]
  Dim s As String

  For Each s In ReturnArrays(sSort, 0)
    Print s
  Next
  For Each s In ReturnArrays(sSort, -1)
    Print s
  Next

End

Private Function ReturnArrays(SortedArray As String[], withNumber As
Boolean) As String[]

  Dim sSingle, sWithNumber As New String[]
  Dim i, n As Integer

  For i = 0 To SortedArray.Max
    ' You can avoid with Tobias's trick (For i = 1 To ...)
    If i < SortedArray.Max Then
      If SortedArray[i] = SortedArray[i + 1] Then
        Inc n
      Else
        Inc n
        sSingle.Push(SortedArray[i])
        sWithNumber.Push(n & SortedArray[i])
        n = 0
      Endif
    Endif
  Next
  Inc n
  sSingle.Push(SortedArray[SortedArray.Max])
  sWithNumber.Push(n & SortedArray[SortedArray.Max])
  If withNumber Then
    Return sWithNumber
  Else
    Return sSingle
  Endif

End

Regards
Gianluigi

2017-06-30 15:05 GMT+02:00 Tobias Boege <[hidden email]>:

> On Fri, 30 Jun 2017, Fernando Cabral wrote:
> > 2017-06-30 7:44 GMT-03:00 Fabien Bodard <[hidden email]>:
> >
> > > The best way is the nando one ... at least for gambas.
> > >
> > > As you have not to matter about what is the index value or the order,
> > > the walk ahead option is the better.
> > >
> > >
> > > Then Fernando ... for big, big things... I think you need to use a DB.
> > > Or a native language.... maybe a sqlite memory structure can be good.
> > >
> >
> > Fabien, since this is a one-time only thing, I don't think I'd be better
> > off witha database.
> > Basically, I read a text file an then break it down into words, sentences
> > and paragraphs.
> > Next I  count the items in each array (words, sentences paragraphs).
> > Array.count works wonderfully.
> > After that, have to eliminate the duplicate words (Array.words). But in
> > doing it, al also have to count
> > how many times each word appeared.
> >
> > Finally I sort the Array.Sentences and the Array.Paragraphs by size
> > (string.len()). The Array.WOrds are
> > sorted by count + lenght. This is all woring good.
> >
> > So, my quest is for the fastest way do eliminate the words duplicates
> while
> > I count them.
> > For the time being, here is a working solution based on system' s sort |
> > uniq:
> >
> > Here is one of the versions I have been using:
> >
> > Exec ["/usr/bin/uniq", "Unsorted.txt", "Sorted.srt2"] Wait
> > Exec ["/usr/bin/uniq", "-ci", "SortedWords.srt2",  SortedWords.srt3"]
> Wait
> > Exec ["/usr/bin/sort", "-bnr", SortedWords.srt3] To UniqWords
> >
>
> Are those temporary files? You can avoid those by piping your data into the
> processes and reading their output directly. Otherwise the Temp$() function
> gives you better temporary files.
>
> > WordArray = split (UniqWords, "\n")
> >
> > So, I end up with the result I want. It's effective. Now, it would be
> more
> > elegant If I could do the same
> > with Gambas. Of course, the sorting would be easy with the builting
> > WordArray.sort ().
> > But how about te '"/usr/bin/uniq", "-ci" ...' part?
> >
>
> I feel like my other mail answered this, but I can give you another version
> of that routine (which I said I would leave as an exercise to you):
>
>   ' Remove duplicates in an array like "uniq -ci". String comparison is
>   ' case insensitive. The i-th entry in the returned array counts how many
>   ' times aStrings[i] (in the de-duplicated array) was present in the
> input.
>   ' The data in ~aStrings~ is overridden. Assumes the array is sorted.
>   Private Function Uniq(aStrings As String[]) As Integer[]
>     Dim iSrc, iLast As Integer
>     Dim aCount As New Integer[](aStrings.Count)
>
>     If Not aStrings.Count Then Return []
>     iLast = 0
>     aCount[iLast] = 1
>     For iSrc = 1 To aStrings.Max
>       If String.Comp(aStrings[iSrc], aStrings[iLast], gb.IgnoreCase) Then
>         Inc iLast
>         aStrings[iLast] = aStrings[iSrc]
>         aCount[iLast] = 1
>       Else
>         Inc aCount[iLast]
>       Endif
>     Next
>
>     ' Now shrink the arrays to the memory they actually need
>     aStrings.Resize(iLast + 1)
>     aCount.Resize(iLast + 1)
>     Return aCount
>   End
>
> What, in my opinion, is at least theoretically better here than the other
> proposed solutions is that it runs in linear time, while nando's is
> quadratic[*]. (Of course, if you sort beforehand, it will become n*log(n),
> which is still better than quadratic.)
>
> Attached is a test script with some words. It runs the sort + uniq
> utilities
> first and then Array.Sort() + the Uniq() function above. The program then
> prints the *diff* between the two outputs. I get an empty diff, meaning
> that
> my Gambas routines produce exactly the same output as the shell utilities.
>
> Regards,
> Tobi
>
> [*] He calls array functions Add() and Find() inside a For loop that runs
>     over an array of size n. Adding elements to an array or searching an
>     array have themselves worst-case linear complexity, giving quadratic
>     overall. My implementation reserves some more space in advance to
>     avoid calling Add() in a loop. Since the array is sorted, we can go
>     without Find(), too. Actually, as you may know, adding an element to
>     the end of an array can be implemented in amortized constant time
>     (as C++'s std::vector does), by wasting space, but AFAICS Gambas
>     doesn't do this, but I could be wrong.
>
> --
> "There's an old saying: Don't change anything... ever!" -- Mr. Monk
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Gambas-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gambas-user
>
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gambas-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gambas-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: I need a hint on how to deleted duplicate items in a array

Tobias Boege-2
On Fri, 30 Jun 2017, Gianluigi wrote:

> What was wrong in my example which meant this?
>
> Public Sub Main()
>
>   Dim sSort As String[] = ["A", "B", "B", "B", "C", "D", "D", "E", "E",
> "E", "E", "F"]
>   Dim s As String
>
>   For Each s In ReturnArrays(sSort, 0)
>     Print s
>   Next
>   For Each s In ReturnArrays(sSort, -1)
>     Print s
>   Next
>
> End
>
> Private Function ReturnArrays(SortedArray As String[], withNumber As
> Boolean) As String[]
>
>   Dim sSingle, sWithNumber As New String[]
>   Dim i, n As Integer
>
>   For i = 0 To SortedArray.Max
>     ' You can avoid with Tobias's trick (For i = 1 To ...)
>     If i < SortedArray.Max Then
>       If SortedArray[i] = SortedArray[i + 1] Then
>         Inc n
>       Else
>         Inc n
>         sSingle.Push(SortedArray[i])
>         sWithNumber.Push(n & SortedArray[i])
>         n = 0
>       Endif
>     Endif
>   Next
>   Inc n
>   sSingle.Push(SortedArray[SortedArray.Max])
>   sWithNumber.Push(n & SortedArray[SortedArray.Max])
>   If withNumber Then
>     Return sWithNumber
>   Else
>     Return sSingle
>   Endif
>
> End
>

I wouldn't say there is anything *wrong* with it, but it also has quadratic
worst-case running time. You use String[].Push() which is just another name
for String[].Add(). Adding an element to an array (the straightforward way)
is done by extending the space of that array by one further element and
storing the value there. But extending the space of an array could potentially
require you to copy the whole array somewhere else (where you have enough
free memory at the end of the array to enlarge it). Doing worst-case analysis,
we have to assume that this bad case always occurs.

If you fill an array with n values, e.g.

  Dim a As New Integer[]
  For i = 1 To n
    a.Add(i)
  Next

then you loop n times and in the i-th iteration there will be already
i-many elements in your array. Adding one further element to it will,
in the worst case, require i copy operations to be performed. 9-year-old
C.F. Gauss will tell you that the amount of store operations is about n^2.

And your function does two jobs simultaneously but only returns the result
of one of the jobs. The output you get is only worth half the time you spent.

Regards,
Tobi

--
"There's an old saying: Don't change anything... ever!" -- Mr. Monk

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gambas-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gambas-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: I need a hint on how to deleted duplicate items in a array

Gianluigi
2017-06-30 17:21 GMT+02:00 Tobias Boege <[hidden email]>:

>
> I wouldn't say there is anything *wrong* with it, but it also has quadratic
> worst-case running time. You use String[].Push() which is just another name
> for String[].Add(). Adding an element to an array (the straightforward way)
> is done by extending the space of that array by one further element and
> storing the value there. But extending the space of an array could
> potentially
> require you to copy the whole array somewhere else (where you have enough
> free memory at the end of the array to enlarge it). Doing worst-case
> analysis,
> we have to assume that this bad case always occurs.
>
> If you fill an array with n values, e.g.
>
>   Dim a As New Integer[]
>   For i = 1 To n
>     a.Add(i)
>   Next
>
> then you loop n times and in the i-th iteration there will be already
> i-many elements in your array. Adding one further element to it will,
> in the worst case, require i copy operations to be performed. 9-year-old
> C.F. Gauss will tell you that the amount of store operations is about n^2.
>
>
Tobias you are always kind and thank you very much.
Is possible for you to explain this more elementarily, for me (a poorly
educated boy :-) )



> And your function does two jobs simultaneously but only returns the result
> of one of the jobs. The output you get is only worth half the time you
> spent.
>
>
I did two functions in one, just to save space, this is a simple example.
:-)

Regards
Gianluigi
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gambas-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gambas-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: I need a hint on how to deleted duplicate items in a array

Gianluigi
Sorry Tobias,
other explanations are not necessary.
I would not be able to understand :-(
I accept what you already explained to me as a dogma and I will try to put
it into practice by copying your code :-).

Thanks again.

Gianluigi

2017-06-30 17:44 GMT+02:00 Gianluigi <[hidden email]>:

>
> 2017-06-30 17:21 GMT+02:00 Tobias Boege <[hidden email]>:
>
>>
>> I wouldn't say there is anything *wrong* with it, but it also has
>> quadratic
>> worst-case running time. You use String[].Push() which is just another
>> name
>> for String[].Add(). Adding an element to an array (the straightforward
>> way)
>> is done by extending the space of that array by one further element and
>> storing the value there. But extending the space of an array could
>> potentially
>> require you to copy the whole array somewhere else (where you have enough
>> free memory at the end of the array to enlarge it). Doing worst-case
>> analysis,
>> we have to assume that this bad case always occurs.
>>
>> If you fill an array with n values, e.g.
>>
>>   Dim a As New Integer[]
>>   For i = 1 To n
>>     a.Add(i)
>>   Next
>>
>> then you loop n times and in the i-th iteration there will be already
>> i-many elements in your array. Adding one further element to it will,
>> in the worst case, require i copy operations to be performed. 9-year-old
>> C.F. Gauss will tell you that the amount of store operations is about n^2.
>>
>>
> Tobias you are always kind and thank you very much.
> Is possible for you to explain this more elementarily, for me (a poorly
> educated boy :-) )
>
>
>
>> And your function does two jobs simultaneously but only returns the result
>> of one of the jobs. The output you get is only worth half the time you
>> spent.
>>
>>
> I did two functions in one, just to save space, this is a simple example.
> :-)
>
> Regards
> Gianluigi
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gambas-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gambas-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: I need a hint on how to deleted duplicate items in a array

Gianluigi
Just for curiosity, on my computer, my function (double) processes 10
million strings (first and last name) in about 3 seconds.
Very naif measurement using Timers and a limited number of names and
surnames eg Willy Weber has come up 11051 times

To demonstrate the goodness of Tobias' arguments, about 1 million 3 cents a
second I really understood (I hope) what he wanted to say.

Sorry my response times but today my modem works worse than my brain.

Regards
Gianluigi

2017-06-30 17:58 GMT+02:00 Gianluigi <[hidden email]>:

> Sorry Tobias,
> other explanations are not necessary.
> I would not be able to understand :-(
> I accept what you already explained to me as a dogma and I will try to put
> it into practice by copying your code :-).
>
> Thanks again.
>
> Gianluigi
>
> 2017-06-30 17:44 GMT+02:00 Gianluigi <[hidden email]>:
>
>>
>> 2017-06-30 17:21 GMT+02:00 Tobias Boege <[hidden email]>:
>>
>>>
>>> I wouldn't say there is anything *wrong* with it, but it also has
>>> quadratic
>>> worst-case running time. You use String[].Push() which is just another
>>> name
>>> for String[].Add(). Adding an element to an array (the straightforward
>>> way)
>>> is done by extending the space of that array by one further element and
>>> storing the value there. But extending the space of an array could
>>> potentially
>>> require you to copy the whole array somewhere else (where you have enough
>>> free memory at the end of the array to enlarge it). Doing worst-case
>>> analysis,
>>> we have to assume that this bad case always occurs.
>>>
>>> If you fill an array with n values, e.g.
>>>
>>>   Dim a As New Integer[]
>>>   For i = 1 To n
>>>     a.Add(i)
>>>   Next
>>>
>>> then you loop n times and in the i-th iteration there will be already
>>> i-many elements in your array. Adding one further element to it will,
>>> in the worst case, require i copy operations to be performed. 9-year-old
>>> C.F. Gauss will tell you that the amount of store operations is about
>>> n^2.
>>>
>>>
>> Tobias you are always kind and thank you very much.
>> Is possible for you to explain this more elementarily, for me (a poorly
>> educated boy :-) )
>>
>>
>>
>>> And your function does two jobs simultaneously but only returns the
>>> result
>>> of one of the jobs. The output you get is only worth half the time you
>>> spent.
>>>
>>>
>> I did two functions in one, just to save space, this is a simple example.
>> :-)
>>
>> Regards
>> Gianluigi
>>
>
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gambas-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gambas-user
12
Loading...