Also, eliminate the 'atom' field of TPpToken.
Parsing a real 300 line shader, through to making the AST, is about 10% faster.
Memory is slightly reduced (< 1%).
The whole google-test suite, inclusive of all testing overhead, SPIR-V generation,
etc., runs 3% faster.
Since this is a code *simplification* that leads to perf. improvement, I'm not
going to invest too much more in measuring the perf. than this. The PP code is
simply now in a better state to see how to further rationalize/improve it.
Removed the preprocesser memory pool.
Removed extra copies and unnecessary allocations of objects related to the ones
that were using the pool.
Replaced some allocated pointers with objects instead, generally using more
modern techiques. There end up being fewer memory allocations/deletions to get right.
Overall combined effect of all changes is to use slightly less memory and
run slightly faster (< 1% for both, but noticable).
As part of simplifying the code base, this change makes it easier to see
PP symbol tracking, which I suspect has an even bigger run-time simplification
to make.
Implement token pasting as per the C++ specification, within the current
style of the PP code.
Non-identifiers (turning 12 ## 10 into the numeral 1210) is not yet covered;
they should be a simple incremental change built on this one.
Addresses issue #255.
Code using atEndOfFile was dead, instead do something useful with
the scanners atEndOfInput(). This allows a better error message
for early termination of cascading errors.
When preprocessing only, some tokens were emitted as <bad token>.
This fixes them to preserve their original content.
This supplants PR #182, with a correction and test results.
This would look ahead for a second #, for token pasting, and if not
found, backup one token. This is fine, unless at the end of line,
which would backup the #, rather than the look ahead.
- Add new keyword int64_t/uint64_t/i64vec/u64vec.
- Support 64-bit integer literals (dec/hex/oct).
- Support built-in operators for 64-bit integer type.
- Add implicit and explicit type conversion for 64-bit integer type.
- Add new built-in functions defined in this extension.
This plumbs both the current file path and the include depth
back up to the includer. This allows the includer to properly
support relative paths.
This also replaces the string copy that was done during include
with a zero-copy method of accomplishing the same thing. This
prevents extra copies of entire files.
Adds parseVersions.h as the base TParseVersions for versioning,
and splits the remainder between TParseContextBase (sharable across parsers)
and TParseContext (now the GLSL-specific part).
There is no calls to the TPpContext that could change the current input,
so every calls to pp->getChar and pp->ungetChar ultimately call this->getch
and this->ungetch while adding overhead of virtual calls and vector::back.
Now extensions required by preprocessor should be checked via
the ppRequireExtensions method. This is more clear and coherent
with the rest of the code.
After parsing a #include directive, we push a TokenizableString
which contains the content of the included file into the input
stack. Henceforth, tokens will be read from the newly pushed
TokenizableString. However, the scanner in TParseContext still
points to the previous input stream. We need to update the scanner
to point to the new input stream inside TokenizableString. Thus,
the setCurrent{String|Line|..} method in TParseContext updates
the status of the correct input stream. After finishing the newly
pushed TokenizableString, we need to restore the scanner to the
previous input stream.
This patch introduces a new extension, GL_GOOGLE_include_directive,
to enable support #include directives. It depends on the extension
GL_GOOGLE_cpp_style_line_directive.
When an include directive is recognized by the preprocessor, it
executes a callback on the filepath argument to obtain the file
contents. That way the compilation client can deal with the file
system, include paths, etc.
Currently only accepts quoted filepaths -- no angle brackets yet.
According to the GLSL spec, the second parameter to #line should be
an integer source string number and __FILE__ will be substituted
with the integer source string number currently processed. This
patch extends the syntax of #line and __FILE__. Now #line accepts
as the second parameter a filename string quoted by double quotation
marks. And if such a #line is set, __FILE__ will be substituted with
the currently set filename string. The implementation is done via
introducing a new extension GL_GOOGLE_cpp_style_line_directive using
the extension framework.
The purpose is to support cpp-style #line directives, which is
required by #include.
Fixes issue #25. (char 255 aliased to -1 and missing tests for end of input).
1) All layers of input scanning now share a single EndOfInput value.
This avoids translation of it across layers of encapsulation.
2) Some places looking for end of line were not stopping on EndOfInput.
3) Use of "char" for the input made char values > 127 be negative numbers.
This allowed for aliasing of 255 to -1, etc. This is fixed by using
unsigned char.
This is just for '\' that's not before a new line.
Note the specification says it has no use other than as line continuation,
but #error is a grey area. (There are no escape sequences.)
