nsftools blog | comments

[1] Charles Robinson @ 11:18AM | 2007-10-13

[meekly raises his hand from the back of the class] Mr. Robichaux, how did you put 75,000 elements into StringBuffer when you have a maxsize of 32,000 in your code? Wouldn't it have to go up to 75,000?


[2] Tim Tripcony @ 12:07PM | 2007-10-13

I'm thinkin' the ideal approach, then, would be partially determined by the reason for the concatenation. For example, in a WebQueryOpen that is generating a bunch of HTML to write to a rich text field, it makes sense to just write each chunk directly to that field instead of keeping everything in a String or a NotesStream and then writing the whole value to the field. If you're generating DXL to import into a database, either a NotesStream or a rich text item can be passed as an input to the importer, so unless there's a reason for storing it in the document as well (which you would if you wanted detailed logging of what you'd imported), it's faster - and slightly less code - just to use a stream. And for massive reporting operations, it's worthwhile to include the extra StringBuffer code, because those minute differences can add up enough to be noticeable.


[3] Julian Robichaux @ 02:37PM | 2007-10-13

@Charles: I can do it because I'm sneaky.
The way the StringBuffer class works, once I hit the upper limit of my internal array I write everything out to a buffer string and reset the array so I can start over.

Also, when I do the reset I try to double the size of the internal array (if possible), much like the Java StringBuffer does. So even if you start with a small internal array size, it still works pretty well once you get into the thousands of iterations.

@Tim: very good point. If you're working with a document already, it makes sense just to use a field on the doc. I hadn't thought about that, and that's probably the situation that Tommy was looking at when he proposed the technique in the first place.

Now, if there was just a faster way to step through a string character-by-character...

Yes, that's a challenge.



[4] Nathan T. Freeman @ 05:08PM | 2007-10-13

You are a god among insects. Never let anyone tell you any different.


[5] Charles Robinson @ 09:14PM | 2007-10-13

Ah.... buffer = Me.toString ... I didn't fully understand what that was doing when I first read through it. You make my head hurt in a good way.


[6] Julian Robichaux @ 10:07PM | 2007-10-13

Nathan, did you just quote Magneto?


[7] Tommy Valand @ 06:47AM | 2007-10-14

@Julian: I was in a WQO/fill-RT-field-with-html-mindset when I wrote my blogpost. So your assumption is quite correct.

Great post btw. I love benchmarks

Regarding character by character, have you tried Java/LS2J?

And since were on the topic of benchmarks, do you have any numbers on ReDim performance?



[8] Julian Robichaux @ 09:15AM | 2007-10-14

Java is definitely faster than LotusScript for string parsing. The weird thing is, there's sometimes a short "startup" delay with Java agents that eliminates the performance advantage in the case of smaller strings. At least there used to be -- haven't tested that particular thing in ages.

But I'll be doing some testing now, and I hope you will too! Charles Robinson already forwarded some code to me that I think is going to be unbeatable.

Regarding ReDim performance, I don't have any official numbers. I do know that it's not quite as bad as people think it is, but you should still try not to ReDim in loops. Better to ReDim a huge array before a loop, and then ReDim Preserve the array back down to its proper size after the loop.

Maybe I'll do some performance testing on that one too... Or you can, of course.



[9] Tommy Valand @ 11:37AM | 2007-10-14



[10] Charles Robinson @ 01:27PM | 2007-10-14

I have to redeem myself after that last dumb comment.

http://cubert-codepoet.blogspot.com/2007/10/julians-parsing-challenge.html

@Nathan - I feel like a blind deaf mute quadriplegic in the company of opera singers and Cirque du Soleil performers.



[11] Nathan T. Freeman @ 04:27PM | 2007-10-14

Yes, I did.


[12] Dwight Wilbanks @ 07:55PM | 2007-10-14

I tried to better your times using copymemory api function and failed misserably. I know that the copymemory function is ultrafast and its the most efficient from the standpoint of moving the string data, but, it has extra lines of code to move around the starting and ending positions, which killed the times.

I did however increase the performance on your by about 10% in my test by passing a string instead of a variant to add text. This is good if you know what you have is definiatly starting out as text, it does not need to be converted to variant upon entry to the sub



[13] Tommy Valand @ 11:33AM | 2007-10-15

Numbers from string concatenation in Java:
Standard concatenation (75 000 concatenations): 192.594s, string length, 300 000.

StringBuffer-concatenation (75 000 concatenations): 0.062s, string length, 300 000.

Code/post:
http://dontpanic82.blogspot.com/2007/10/string-concatenation-with-java-agent.html



[14] Dmytro Pastovenskyi @ 08:43AM | 2007-10-25

great article. you know, I have never had this problem... I mean so many operation of concatenation. But in any case "researching" is great !

Thank you.



[15] Dmytro Pastovenskyi @ 09:25AM | 2007-10-25

by the way,

I have tested another approach, and it gave me good result.

Dim s As String
Dim startTime As Single
Dim i As Long

Dim wording As String
Let wording = "test"
s = ""
Dim max As Long
max = 10000

startTime = Timer()

Dim sb As New StringBuffer(16)
For i = 0 To max
sb.append(wording)
Next
s = sb.toString
Print "Final string buffer length = " & Len(s) & ". Timer = " & (Timer() - startTime) & " sec."

Dim d As String * 16000
startTime = Timer()
For i = 0 To max
d = d & wording
Next
Print "Final string length = " & Len(s) & ". Timer = " & (Timer() - startTime) & " sec."


But the problem is that we can free for 1 string variable only 16k.



add a comment

name:
e-mail:
www:

HTML markup is not allowed in your comments, although URLs will be automatically converted to links (make sure there is a space before and after the URL).