-
- Downloads
Add LLStringUtil::getTokens() overload handling quoted substrings.
We didn't have any tokenizer suitable for scanning something like a bash command line. We do have a couple hacks, e.g. LLExternalEditor::tokenize() and LLCommandLineParser::parseCommandLineString(). Both try to work around boost::tokenizer limitations; but existing boost::tokenizer support just doesn't address this case. Neither of the above is available as a general scanner anyway, and parseCommandLineString() fails outright when passed "". New getTokens() also distinguishes between "drop delimiters" (e.g. space, return, newline) to be discarded from the token stream, versus "keep delimiters" (e.g. "+-*/") to be returned as tokens in their own right. There's an overload that honors escapes and a more efficient one that doesn't; each has a convenience overload that returns the scanned string vector rather than requiring a separate declaration. Tweak and comment older getTokens() implementation. Add unit tests for both old and new getTokens() implementations. Break out StringVec and std::ostream << StringVec from indra/llcommon/tests/listener.h to StringVec.h: that's coming in handy for a number of different TUT test sources.
Showing
- indra/llcommon/llstring.cpp 9 additions, 7 deletionsindra/llcommon/llstring.cpp
- indra/llcommon/llstring.h 353 additions, 2 deletionsindra/llcommon/llstring.h
- indra/llcommon/tests/StringVec.h 37 additions, 0 deletionsindra/llcommon/tests/StringVec.h
- indra/llcommon/tests/listener.h 3 additions, 18 deletionsindra/llcommon/tests/listener.h
- indra/llcommon/tests/llstring_test.cpp 115 additions, 0 deletionsindra/llcommon/tests/llstring_test.cpp
Loading
Please register or sign in to comment