After a bit more investigation, I found that if you replace the following code:
result.append(begin == eos ? "" : String(cs[begin..<end.successor()]))
with this:
if begin == eos {
result.append("")
} else if let str = String(cs[begin..<end.successor()]) {
result.append(str)
}
runtime goes down from ~3 seconds to ~2.2 seconds.
This is due to a rather insidious API design decision:
init?(_ view: String.UTF16View)
constructs a string out of a UTF16 view, but it can fail. If used in a context where its type is inferred to be non-nullable, the following generic reflection-related init is used instead:
init<T>(_ instance: T)
I'm going to bring this up on the list and see if there are better ways of doing things.
As far as I can tell most of the rest of the time is spent in the Swift native Unicode --> UTF16 decoding machinery, and NSCharacterSet.
OK I found the cause of that weirdness. I had slightly changed my test code since I posted it here weeks ago. After undoing that change I'm seeing the same thing you do.
But this raises more questions, because what I changed is the test code generation. I'm now generating a million different strings instead of adding the same string a million times (note the \(i) at the end of the string):
func generateTestData() -> [String] {
var a = [String]()
for i in 0..<N {
a.append(",,abc, 123 ,x, , more more more,\u{A0}and yet more, \(i)")
}
return a
}
The running time of generateTestData() isn't what we measure but apparently the performance improvement you found only works if the same string is used every time. Otherwise performance drops.
One thing I've noticed is that performing the string scanning operation is relatively cheap. (If the splitAndTrim code is modified to not use Strings and to return a [String.UTF16View], the runtime is around 1.2 seconds.) It's the process of building Strings out of those UTF16 views that is destroying performance.
I still don't know why changing the way the input data are constructed would have that effect, except to guess that the underlying representation is different somehow. I'll file a ticket.
This looks to me like memory allocation / reference counting is at least part of the problem. Slicing a UTF16View to get another UTF16View mostly likely doesn't involve any dynamic memory allocation at all.
This is due to a rather insidious API design decision:
constructs a string out of a UTF16 view, but it can fail. If used in a context where its type is inferred to be non-nullable, the following generic reflection-related init is used instead: I'm going to bring this up on the list and see if there are better ways of doing things.As far as I can tell most of the rest of the time is spent in the Swift native Unicode --> UTF16 decoding machinery, and NSCharacterSet.