From 3add402da9fc5b574f34e37e951779212ce28ed1 Mon Sep 17 00:00:00 2001 From: Kyle Gunger Date: Sun, 22 Aug 2021 15:25:54 -0400 Subject: Begin port of tnsl-parse code to native TNSL --- tnslc/src/parse/tokenizer.tnsl | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 tnslc/src/parse/tokenizer.tnsl (limited to 'tnslc/src/parse/tokenizer.tnsl') diff --git a/tnslc/src/parse/tokenizer.tnsl b/tnslc/src/parse/tokenizer.tnsl new file mode 100644 index 0000000..ec34d83 --- /dev/null +++ b/tnslc/src/parse/tokenizer.tnsl @@ -0,0 +1,25 @@ +/# + Copyright 2021 Kyle Gunger + + This file is licensed under the CDDL 1.0 (the License) + and may only be used in accordance with the License. + You should have received a copy of the License with this + software/source code. If you did not, a copy can be found + at the following URL: + + https://opensource.org/licenses/CDDL-1.0 + + THIS SOFTWARE/SOURCE CODE IS PROVIDED "AS IS" WITH NO + WARRANTY, GUARANTEE, OR CLAIM OF FITNESS FOR ANY PURPOSE + EXPRESS OR IMPLIED +#/ + + +/## + parse.numeric_literal tokenizes the next numeric literal value in a file. + Returns a token with the proper data as well as the number of characters read + +#; numeric_literal () [Token, uint] + + +;/ \ No newline at end of file -- cgit v1.2.3