diff options
| author | Kyle Gunger <kgunger12@gmail.com> | 2021-08-22 15:25:54 -0400 |
|---|---|---|
| committer | Kyle Gunger <kgunger12@gmail.com> | 2021-08-22 15:25:54 -0400 |
| commit | 3add402da9fc5b574f34e37e951779212ce28ed1 (patch) | |
| tree | a081a2331923dce6ee29c87285be901d58b0bc4b /tnslc/src/parse/tokenizer.tnsl | |
| parent | 11f9c56ae3861e32ac45785e9f30ed5f4c19ea32 (diff) | |
Begin port of tnsl-parse code to native TNSL
Diffstat (limited to 'tnslc/src/parse/tokenizer.tnsl')
| -rw-r--r-- | tnslc/src/parse/tokenizer.tnsl | 25 |
1 files changed, 25 insertions, 0 deletions
diff --git a/tnslc/src/parse/tokenizer.tnsl b/tnslc/src/parse/tokenizer.tnsl new file mode 100644 index 0000000..ec34d83 --- /dev/null +++ b/tnslc/src/parse/tokenizer.tnsl @@ -0,0 +1,25 @@ +/# + Copyright 2021 Kyle Gunger + + This file is licensed under the CDDL 1.0 (the License) + and may only be used in accordance with the License. + You should have received a copy of the License with this + software/source code. If you did not, a copy can be found + at the following URL: + + https://opensource.org/licenses/CDDL-1.0 + + THIS SOFTWARE/SOURCE CODE IS PROVIDED "AS IS" WITH NO + WARRANTY, GUARANTEE, OR CLAIM OF FITNESS FOR ANY PURPOSE + EXPRESS OR IMPLIED +#/ + + +/## + parse.numeric_literal tokenizes the next numeric literal value in a file. + Returns a token with the proper data as well as the number of characters read + +#; numeric_literal () [Token, uint] + + +;/
\ No newline at end of file |