This fixes the comparison in macro body redefinitions, where initial
white-space differences do not matter, but internal white-space differences
do matter.
This has been a continually fragile area. Switching to a vector of real
objects, instead of a linearized stream of characters, removes a bunch of
code and generally makes this area more robust.
Fixes#1228. Fixes#234.
This uses imbue() to be locale independent. Notes:
- 'sstream >> double' is much slower than strtod()
* this was measurable in the test suite as a whole, despite being
a tiny fraction of what the test suite does
- so, this embeds a fast path that bypasses sstream most of the time
=> the test suite is faster than before
- sstream is probably slower, because it does more accurate rounding than strtod()
- sstream does not create INFINITY by itself, this was done based on failure inferencing
- fixed ParseHelper.cpp newlines (crlf -> lf)
- removed trailing white space in most source files
- fix some spelling issues
- extra blank lines
- tabs to spaces
- replace #include comment about no location
Also, eliminate the 'atom' field of TPpToken.
Parsing a real 300 line shader, through to making the AST, is about 10% faster.
Memory is slightly reduced (< 1%).
The whole google-test suite, inclusive of all testing overhead, SPIR-V generation,
etc., runs 3% faster.
Since this is a code *simplification* that leads to perf. improvement, I'm not
going to invest too much more in measuring the perf. than this. The PP code is
simply now in a better state to see how to further rationalize/improve it.
Removed the preprocesser memory pool.
Removed extra copies and unnecessary allocations of objects related to the ones
that were using the pool.
Replaced some allocated pointers with objects instead, generally using more
modern techiques. There end up being fewer memory allocations/deletions to get right.
Overall combined effect of all changes is to use slightly less memory and
run slightly faster (< 1% for both, but noticable).
As part of simplifying the code base, this change makes it easier to see
PP symbol tracking, which I suspect has an even bigger run-time simplification
to make.
Implement token pasting as per the C++ specification, within the current
style of the PP code.
Non-identifiers (turning 12 ## 10 into the numeral 1210) is not yet covered;
they should be a simple incremental change built on this one.
Addresses issue #255.
- Add new keyword int64_t/uint64_t/i64vec/u64vec.
- Support 64-bit integer literals (dec/hex/oct).
- Support built-in operators for 64-bit integer type.
- Add implicit and explicit type conversion for 64-bit integer type.
- Add new built-in functions defined in this extension.
This plumbs both the current file path and the include depth
back up to the includer. This allows the includer to properly
support relative paths.
This also replaces the string copy that was done during include
with a zero-copy method of accomplishing the same thing. This
prevents extra copies of entire files.
Adds parseVersions.h as the base TParseVersions for versioning,
and splits the remainder between TParseContextBase (sharable across parsers)
and TParseContext (now the GLSL-specific part).
After parsing a #include directive, we push a TokenizableString
which contains the content of the included file into the input
stack. Henceforth, tokens will be read from the newly pushed
TokenizableString. However, the scanner in TParseContext still
points to the previous input stream. We need to update the scanner
to point to the new input stream inside TokenizableString. Thus,
the setCurrent{String|Line|..} method in TParseContext updates
the status of the correct input stream. After finishing the newly
pushed TokenizableString, we need to restore the scanner to the
previous input stream.
When an include directive is recognized by the preprocessor, it
executes a callback on the filepath argument to obtain the file
contents. That way the compilation client can deal with the file
system, include paths, etc.
Currently only accepts quoted filepaths -- no angle brackets yet.
Fixes issue #25. (char 255 aliased to -1 and missing tests for end of input).
1) All layers of input scanning now share a single EndOfInput value.
This avoids translation of it across layers of encapsulation.
2) Some places looking for end of line were not stopping on EndOfInput.
3) Use of "char" for the input made char values > 127 be negative numbers.
This allowed for aliasing of 255 to -1, etc. This is fixed by using
unsigned char.
It also removes some old code that ancient compilers used to need.
However, the main issue is getting access to hash functions for
unordered_map in portable way.
This simplification is a prelude to eliminating what I appear unnecessary
symbol inserts into tables when tokenizing in the preprecessor, which
show up as taking notable time. (Performance issue.) It also simply makes
the preprocessor easier to understand, which it is badly in need of.
