Adding support for new syntax (even just to add to the ast and parser) is pretty involved. Here is a sketch example which may be out of date. Please get in touch if you are trying to follow this and have any questions.
Worked through example of adding support for parsing data types with two precision components. (This is already in hssqlppp).
The parser/syntax currently supports the precision of floating point types, e.g.
create table t { col float(24); }
We want it to also support two components, for numeric types:
create table t { col numeric(5,3); }
add test(s) to the automated tests under src-extra/tests/
compile and run the tests to check the new test fails
add new datatypes to AstInternal.ag
fix any default uuagc rules if needed
run the make target which runs uuagc to generate the hs from the ag files
add code for parsing and pretty printing the new syntax
compile and run the tests to check that the new test(s) now pass
The first step: add a test. At the moment, all the parsing tests go src-extra/tests/Database/HsSqlPpp/Tests/ParserTest.lhs. You could also add a new module, and add it to src-extra/tests/Tests.
To add the new test, you have to figure out how the new abstract syntax will look. We can defer this while you add the test, until we actually edit the abstract syntax types. The code added to ParserTests.lhs:
e "245.1::numeric(5,3)" (Cast ea (FloatLit ea 245.1) (PrecTypeName ea "numeric" 5))
Now check the test fails.
$ make src-extra/tests/Tests
$ src-extra/tests/Tests -t "numeric"
parserTests:
parse expressions:
simple operators:
parse 245.34::numeric(5,3): [Failed]
Failed: (line 1, column 7):
unexpected ((line 1, column 7),SymbolTok "::")
expecting operator or end of input
:1:7:
Context:
245.34::numeric(5,3)
^
ERROR HERE
Test Cases Total
Passed 0 0
Failed 1 1
Total 1 1
$
You can use -t [string] argument to the tests executable to only run the tests which match the string.
Editing syntax is currently pretty hairy in hssqlppp. The first place to go is the AstInternal.ag file, which is not written in Haskell, but in uuagc.
Searching for PrecTypeName in this file should give a good place to alter the syntax.
You get a block like this:
DATA TypeName | SimpleTypeName ann:Annotation tn:String | PrecTypeName ann:Annotation tn:String prec:Integer | ArrayTypeName ann:Annotation typ:TypeName | SetOfTypeName ann:Annotation typ:TypeName
Add a new Prec2TypeName constructor with two integer args:
DATA TypeName | SimpleTypeName ann:Annotation tn:String | PrecTypeName ann:Annotation tn:String prec:Integer | Prec2TypeName ann:Annotation tn:String prec:Integer prec1:Integer | ArrayTypeName ann:Annotation typ:TypeName | SetOfTypeName ann:Annotation typ:TypeName
Now you have made changes to an ag file, we need to run uuagc to produce AstInternal.hs.
You can use the makefile to do this.
$ make src/Database/HsSqlPpp/Internals/AstInternal.hs
./TypeChecking/Misc.ag:15:18: warning: Missing rule for synthesized attribute namedType in alternative Prec2TypeName of nonterminal TypeName.
OK, so this is a bit weird. One way to fix it is as follows: copy the style of the PrecTypeName part. The previous code:
SEM TypeName | SimpleTypeName ArrayTypeName SetOfTypeName PrecTypeName lhs.namedType = etmt @loc.tpe lhs.annotatedTree = addTypeErrors (tes @loc.tpe) @loc.backTree SEM TypeName | SimpleTypeName loc.tpe = catLookupType @lhs.cat $ canonicalizeTypeName @tn loc.backTree = SimpleTypeName @ann @tn | ArrayTypeName loc.tpe = lmt @typ.namedType >>= Right . ArrayType loc.backTree = ArrayTypeName @ann @typ.annotatedTree | SetOfTypeName loc.tpe = lmt @typ.namedType >>= Right . SetOfType loc.backTree = SetOfTypeName @ann @typ.annotatedTree | PrecTypeName loc.tpe = catLookupType @lhs.cat $ canonicalizeTypeName @tn loc.backTree = PrecTypeName @ann @tn @prec
And the updated code:
SEM TypeName | SimpleTypeName ArrayTypeName SetOfTypeName PrecTypeName Prec2TypeName lhs.namedType = etmt @loc.tpe lhs.annotatedTree = addTypeErrors (tes @loc.tpe) @loc.backTree SEM TypeName | SimpleTypeName loc.tpe = catLookupType @lhs.cat $ canonicalizeTypeName @tn loc.backTree = SimpleTypeName @ann @tn | ArrayTypeName loc.tpe = lmt @typ.namedType >>= Right . ArrayType loc.backTree = ArrayTypeName @ann @typ.annotatedTree | SetOfTypeName loc.tpe = lmt @typ.namedType >>= Right . SetOfType loc.backTree = SetOfTypeName @ann @typ.annotatedTree | PrecTypeName loc.tpe = catLookupType @lhs.cat $ canonicalizeTypeName @tn loc.backTree = PrecTypeName @ann @tn @prec | Prec2TypeName loc.tpe = catLookupType @lhs.cat $ canonicalizeTypeName @tn loc.backTree = Prec2TypeName @ann @tn @prec @prec1
Now uuagc runs without warnings. You can try and skip fixing these warnings just to get the parsing working. (Patches which add new syntax/parsing but leave the type checker broken are welcome.)
If make completes successfully, then the .ag files should be compiled to hs, and you can then run cabal build to check the compilation.
Time to fix the tests: editing the line in ParserTests.lhs to match the new syntax gives:
e "(245.1)::numeric(5,3)" (Cast ea (FloatLit ea 245.1) (Prec2TypeName ea "numeric" 5 3))
Recompiling works, and running the test fails. You could also use undefined here.
The next job is to add to the parser. The parsing code is mostly in src/Database/HsSqlPpp/Parsing/ParserInternal.lhs.
Loading this file and searching for PrecTypeName gives the typeName parser function:
> typeName :: SParser TypeName > typeName = > choice [ > SetOfTypeName <$> pos <*> (keyword "setof" *> typeName) > ,otherTypeName] > where > otherTypeName = do > p <- pos > s <- map toLower <$> pTypeNameString > choice [PrecTypeName p s <$> parens integer > ,arrayTypeName p s > ,return $ SimpleTypeName p s] > arrayTypeName p s = ArrayTypeName p (SimpleTypeName p s) > <$ symbol "[" <* symbol "]" > --todo: add special cases for the other type names with spaces in them > pTypeNameString = ("double precision" <$ try (keyword "double" > <* keyword "precision")) > <|> idString
Here is the updated version. It could probably be a bit better written - once the tests pass it is easier to can refactor to something clean and elegant.
> typeName :: SParser TypeName > typeName = > choice [ > SetOfTypeName <$> pos <*> (keyword "setof" *> typeName) > ,otherTypeName] > where > otherTypeName = do > p <- pos > s <- map toLower <$> pTypeNameString > choice [try (Prec2TypeName p s > <$> (symbol "(" *> integer) > <*> (symbol "," *> integer <* symbol ")")) > ,PrecTypeName p s <$> parens integer > ,arrayTypeName p s > ,return $ SimpleTypeName p s] > arrayTypeName p s = ArrayTypeName p (SimpleTypeName p s) > <$ symbol "[" <* symbol "]" > --todo: add special cases for the other type names with spaces in them > pTypeNameString = ("double precision" <$ try (keyword "double" > <* keyword "precision")) > <|> idString
Compiling and running the tests gives:
parserTests:
parse expressions:
simple operators:
parse 245.1::numeric(5,3): [Failed]
ERROR: src/Database/HsSqlPpp/Pretty.lhs:(565,2)-(568,67): Non-exhaustive patterns in function convTypeName
Test Cases Total
Passed 0 0
Failed 1 1
Total 1 1
The issue here is that the pretty printer hasn't also been updated. The parsing tests all first parse the string given, check it is the same as the ast given, then pretty print and reparse the pretty printed string to check it still gives the same result. This is a reasonably good way of checking the pretty printing routines, and that parse then pretty print gives the same sql (and pretty print then parse gives the same ast).
The pretty printing routines are in 'src/Database/HsSqlPpp.Pretty'. Here is the line I added after the PrecTypeName pretty printer:
> typeName (PrecTypeName _ s i) = text s <> parens(integer i) > typeName (Prec2TypeName _ s i i1) = text s <> parens (hcatCsv [integer i, integer i1])
Now the test passes and the only thing left is to run all the tests to check nothing else has been broken.