Commit graph

64 commits

Author SHA1 Message Date
John Kessenich
9a2733a978 PP, nonfunctional: Properly encapsulate a TokenStream. 2017-02-10 18:03:01 -07:00
John Kessenich
b49bb2ca5c PP, nonfunctional: Remove crufty bit-twiddling of tokens. 2017-02-10 13:03:40 -07:00
John Kessenich
ce80197c22 Merge pull request #663 from KhronosGroup/full-include-semantics
Includer interface change to support full include semantics (requires downstream changes)
2017-01-10 18:49:07 -07:00
Alex Szpakowski
5f316d9b17 Reorder initializer fields to match variable declaration order (or vice versa) for several class constructors. 2017-01-08 18:21:17 -04:00
John Kessenich
2508602541 PP #include non-functional: consistent notation for "header" and "header name".
This is versus a variety of file-related language, designated, requested, etc.
Will simplify diffs for next commit.
2017-01-06 14:54:18 -07:00
John Kessenich
927608b393 Non-functional: White space after "//", mostly for copyrights. 2017-01-06 12:34:14 -07:00
John Kessenich
ecba76fe73 Non-Functional: Whitespace, comments, replace accidentally deleted comment.
- fixed ParseHelper.cpp newlines (crlf -> lf)
- removed trailing white space in most source files
- fix some spelling issues
- extra blank lines
- tabs to spaces
- replace #include comment about no location
2017-01-06 11:24:14 -07:00
John Kessenich
affc26674d PP: Recognize <> style #include header names. I.e., #include <header-name>.
Also correctly test and handle missing newline.
2017-01-03 11:05:16 -07:00
John Kessenich
f48faec3ee PP: Non-functional: Make a proper class out of the atom <-> string mapping. 2016-12-21 13:49:16 -07:00
Henrik Rydgård
9a931b39bc Fix a large number of warnings about inconsistent usage of 'override' produced by clang 2016-12-21 12:48:08 +01:00
John Kessenich
907aabb6b0 PP: Non-functional: Only use string <-> atom mapping when needed.
Also, eliminate the 'atom' field of TPpToken.

Parsing a real 300 line shader, through to making the AST, is about 10% faster.

Memory is slightly reduced (< 1%).

The whole google-test suite, inclusive of all testing overhead, SPIR-V generation,
etc., runs 3% faster.

Since this is a code *simplification* that leads to perf. improvement, I'm not
going to invest too much more in measuring the perf. than this. The PP code is
simply now in a better state to see how to further rationalize/improve it.
2016-12-20 23:36:05 -07:00
John Kessenich
54af2de761 PP: Non-functional: rationalize TPpToken.
Always keep 'token' outside.
Always return the string to upper levels inside.
2016-12-20 19:42:53 -07:00
John Kessenich
1fbb9c1430 PP: Non-functional: clean up, simplify, completely identical operation. 2016-12-20 18:36:49 -07:00
John Kessenich
b8387c87d0 PP: Non-functional: Remove custom allocator and related improvements.
Removed the preprocesser memory pool.

Removed extra copies and unnecessary allocations of objects related to the ones
that were using the pool.

Replaced some allocated pointers with objects instead, generally using more
modern techiques. There end up being fewer memory allocations/deletions to get right.

Overall combined effect of all changes is to use slightly less memory and
run slightly faster (< 1% for both, but noticable).

As part of simplifying the code base, this change makes it easier to see
PP symbol tracking, which I suspect has an even bigger run-time simplification
to make.
2016-12-19 21:57:06 -07:00
John Kessenich
bfff871dad PP: Add missing i64val code.
Also, checking both 'atom' and 'name' is redundant, and I'm hoping to
eliminate more atom stuff.
2016-12-19 17:46:20 -07:00
John Kessenich
d485e0b710 PP: Implement token pasting for PP identifiers.
Implement token pasting as per the C++ specification, within the current
style of the PP code.
Non-identifiers (turning 12 ## 10 into the numeral 1210) is not yet covered;
they should be a simple incremental change built on this one.
Addresses issue #255.
2016-12-19 09:19:43 -07:00
John Kessenich
31a51becd2 Fix issue #391: input stack underflow on too few macro arguments. 2016-07-19 14:32:52 -06:00
John Kessenich
7f349c73db Build: Remove causes of pedantic warnings. Addresses issue #352 and PR #242. 2016-07-08 22:09:10 -06:00
Rex Xu
8ff43de891 Implement the extension GL_ARB_gpu_shader_int64
- Add new keyword int64_t/uint64_t/i64vec/u64vec.
- Support 64-bit integer literals (dec/hex/oct).
- Support built-in operators for 64-bit integer type.
- Add implicit and explicit type conversion for 64-bit integer type.
- Add new built-in functions defined in this extension.
2016-04-30 13:34:34 +08:00
John Kessenich
1c7e70763b Merge branch 'master' into hlsl-frontend 2016-04-03 20:36:48 -06:00
Andrew Woloszyn
272afd2d0a Fixed compilation issue introduced by my last commit 2016-03-21 16:05:47 -04:00
Andrew Woloszyn
a132af5b78 Updated the includer interface to allow relative includes.
This plumbs both the current file path and the include depth
back up to the includer. This allows the includer to properly
support relative paths.

This also replaces the string copy that was done during include
with a zero-copy method of accomplishing the same thing. This
prevents extra copies of entire files.
2016-03-21 10:19:45 -04:00
John Kessenich
b3dc3acd59 Refactor TParseContext into 3 level inheritance.
Adds parseVersions.h as the base TParseVersions for versioning,
and splits the remainder between TParseContextBase (sharable across parsers)
and TParseContext (now the GLSL-specific part).
2016-03-12 19:08:55 -07:00
baldurk
d7c5ead6a1 Compile fixes for gcc -Wall 2015-08-05 21:04:23 +02:00
David Neto
1ee1f88b40 Remove unused members from TPpContext.
The ErrMsg and ifloc members were unused or had no effect.
2015-08-05 13:15:27 -04:00
Lei Zhang
2b4ebbb1e1 Update scanner in TParseContext when changing the input stream.
After parsing a #include directive, we push a TokenizableString
which contains the content of the included file into the input
stack. Henceforth, tokens will be read from the newly pushed
TokenizableString. However, the scanner in TParseContext still
points to the previous input stream. We need to update the scanner
to point to the new input stream inside TokenizableString. Thus,
the setCurrent{String|Line|..} method in TParseContext updates
the status of the correct input stream. After finishing the newly
pushed TokenizableString, we need to restore the scanner to the
previous input stream.
2015-07-31 19:41:19 -04:00
Dejan Mircevski
7be4b8282d Add #include processing to glslang (though turned off by default).
When an include directive is recognized by the preprocessor, it
executes a callback on the filepath argument to obtain the file
contents.  That way the compilation client can deal with the file
system, include paths, etc.

Currently only accepts quoted filepaths -- no angle brackets yet.
2015-07-31 19:41:18 -04:00
John Kessenich
c777fc2c4c Scanning: 1) rationalize end-of-input everywhere, 2) prevent infinite loop at end-of-input, 3) use positive chars.
Fixes issue #25. (char 255 aliased to -1 and missing tests for end of input).

1) All layers of input scanning now share a single EndOfInput value.
This avoids translation of it across layers of encapsulation.

2) Some places looking for end of line were not stopping on EndOfInput.

3) Use of "char" for the input made char values > 127 be negative numbers.
This allowed for aliasing of 255 to -1, etc.  This is fixed by using
unsigned char.
2015-07-27 09:37:55 -06:00
John Kessenich
f330f08129 Consolidate all token length limits (1024), including one bad one (80) hardcoded on the stack (issue #40, issue #41). 2015-07-25 14:02:53 -06:00
John Kessenich
1056110c30 Perf: Put in a portable std::hash that works with the pool. This turns on the 10-15% perf gain. 2015-07-20 23:59:32 -06:00
John Kessenich
68965c0fa9 Fix g++ build break (portibility problems with stl hash). This partly turns off the stl improvements.
It also removes some old code that ancient compilers used to need.
However, the main issue is getting access to hash functions for
unordered_map in portable way.
2015-07-20 02:06:22 -06:00
John Kessenich
6ab3d582d6 Preprocessor: 1) Rationalize the "fixed atom" scheme, 2) remove redundant lookups when the text is already available.
This simplification is a prelude to eliminating what I appear unnecessary
symbol inserts into tables when tokenizing in the preprecessor, which
show up as taking notable time. (Performance issue.)  It also simply makes
the preprocessor easier to understand, which it is badly in need of.
2015-07-19 22:59:24 -06:00
John Kessenich
2f273369e4 Performance: Simple upgrade/cleanup of stl c++11 containers (10% perf. increase). 2015-07-19 12:03:51 -06:00
John Kessenich
b329715caf Fix clang static analyzer issues, as reported by floooh. 2015-07-11 18:01:03 -06:00
Andre Weissflog
a5ade51363 Fix most clang warnings
- member initializing order in some constructors
- missing default branches in switch-case
- uninitialized variable if switch-case default (uncritical because
  program would exit)
- && and || brace warnings in if()
2015-07-11 14:45:57 +02:00
John Kessenich
47632b7aaf glslang front-end: track column numbers (they don't go anywhere yet, just get tracked). Andrew Woloszyn <awoloszyn@google.com>.
git-svn-id: https://cvs.khronos.org/svn/repos/ogl/trunk/ecosystem/public/sdk/tools/glslang@31506 e7fa87d3-cd2b-0410-9028-fcbf551c1848
2015-06-16 19:49:22 +00:00
John Kessenich
b06127c513 glslang PP: make decimal number parser clearer and add boundary tests. Also, disable the warning for "do {} while (true);"
git-svn-id: https://cvs.khronos.org/svn/repos/ogl/trunk/ecosystem/public/sdk/tools/glslang@31215 e7fa87d3-cd2b-0410-9028-fcbf551c1848
2015-05-16 01:15:41 +00:00
John Kessenich
93dfbe1309 glslang: Fix over 100 warnings from MSVC warning level 4.
git-svn-id: https://cvs.khronos.org/svn/repos/ogl/trunk/ecosystem/public/sdk/tools/glslang@31202 e7fa87d3-cd2b-0410-9028-fcbf551c1848
2015-05-15 17:30:55 +00:00
John Kessenich
777cb2bfd8 Fix linux build error from last check in.
git-svn-id: https://cvs.khronos.org/svn/repos/ogl/trunk/ecosystem/public/sdk/tools/glslang@27836 e7fa87d3-cd2b-0410-9028-fcbf551c1848
2014-08-24 18:23:24 +00:00
John Kessenich
a7a68a9b32 Non-functional misc. changes. Slight increase in performance from moving two performance path methods into a header.
git-svn-id: https://cvs.khronos.org/svn/repos/ogl/trunk/ecosystem/public/sdk/tools/glslang@27835 e7fa87d3-cd2b-0410-9028-fcbf551c1848
2014-08-24 18:21:00 +00:00
John Kessenich
cf0206c341 Restore r26245 and all tests and test results since 4/1:
r26245: Remove "not yet complete" warning for versions 130 and 140, after tidying up a bit and adding tests.  Token pasting is the only known missing feature, and gives its own message now on use.


git-svn-id: https://cvs.khronos.org/svn/repos/ogl/trunk/ecosystem/public/sdk/tools/glslang@26226 e7fa87d3-cd2b-0410-9028-fcbf551c1848
2014-04-14 15:46:40 +00:00
John Kessenich
6494baf371 Check-in a couple submitted char* string portability bug fixes: correct sizing of buffers and stop overwriting preprocessor token names with themselves.
git-svn-id: https://cvs.khronos.org/svn/repos/ogl/trunk/ecosystem/public/sdk/tools/glslang@25402 e7fa87d3-cd2b-0410-9028-fcbf551c1848
2014-02-19 00:08:59 +00:00
John Kessenich
51cdd90fa8 Fix the few non-portable uses of "char" (where a -1 might be relevant): All uses of char are now either "int", "unsigned char" or char arrays for storing strings. Also, went to consistent "char* foo" coding convention. (There were only a few ambiguous uses.)
git-svn-id: https://cvs.khronos.org/svn/repos/ogl/trunk/ecosystem/public/sdk/tools/glslang@25400 e7fa87d3-cd2b-0410-9028-fcbf551c1848
2014-02-18 23:37:57 +00:00
John Kessenich
bd306b89a4 fix g++ complaint and update binaries
git-svn-id: https://cvs.khronos.org/svn/repos/ogl/trunk/ecosystem/public/sdk/tools/glslang@24627 e7fa87d3-cd2b-0410-9028-fcbf551c1848
2013-12-30 20:40:03 +00:00
John Kessenich
fcb4ed054c Preprocessor: Rationalize, simplify, and correct the stack of input sources and their tokenization. This included
- consistently dealing with EOF and its effect on error recovery (bug 11444, #1)
 - turning a simulated OO hierarchy of function pointers and typecasting into a real C++ class hierarchy
 - correctly handling '\' everywhere, in all classes of tokens, as a layer before preprocessing itself
 - conditionally handling '\n' in macro expansion input, depending on whether it is in a macro expression or not
 - delete some unused "memory cleanup" code


git-svn-id: https://cvs.khronos.org/svn/repos/ogl/trunk/ecosystem/public/sdk/tools/glslang@24626 e7fa87d3-cd2b-0410-9028-fcbf551c1848
2013-12-30 20:34:28 +00:00
John Kessenich
2d2f31625d Preprocessor: Implement skipping of macros that evaluate to nothing while evaluating expressions. They had been causing early termination of an expression's evaluation.
Also includes some unrelated updates of documentation.


git-svn-id: https://cvs.khronos.org/svn/repos/ogl/trunk/ecosystem/public/sdk/tools/glslang@24420 e7fa87d3-cd2b-0410-9028-fcbf551c1848
2013-12-10 00:25:14 +00:00
John Kessenich
7fc3669195 Preprocessor: Add short-circuit logic so ES errors are not given on undefined variables that are short circuited in && and || preprocessing expressions.
git-svn-id: https://cvs.khronos.org/svn/repos/ogl/trunk/ecosystem/public/sdk/tools/glslang@24390 e7fa87d3-cd2b-0410-9028-fcbf551c1848
2013-12-06 18:19:22 +00:00
John Kessenich
e28beee891 Preprocessor: Fixed that some comments incorrectly substituted a new line instead of space. Also generally cleaned up the space-related coding.
git-svn-id: https://cvs.khronos.org/svn/repos/ogl/trunk/ecosystem/public/sdk/tools/glslang@24387 e7fa87d3-cd2b-0410-9028-fcbf551c1848
2013-12-06 16:13:47 +00:00
John Kessenich
66cdf36521 Detect and give error on second occurrence of #version.
git-svn-id: https://cvs.khronos.org/svn/repos/ogl/trunk/ecosystem/public/sdk/tools/glslang@24347 e7fa87d3-cd2b-0410-9028-fcbf551c1848
2013-12-04 20:08:16 +00:00
John Kessenich
e1f0f5b31f Re-allow non-language characters in ES 100 comments, including '\', except for '\' just before a new line.
git-svn-id: https://cvs.khronos.org/svn/repos/ogl/trunk/ecosystem/public/sdk/tools/glslang@24331 e7fa87d3-cd2b-0410-9028-fcbf551c1848
2013-12-04 17:23:03 +00:00