You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It would be super cool if there was a JSON schema for hpack, which would automatically enable IDE tools to provide autocompletion and checking of hpack files. I'm motivated for this by my terrible memory for the syntax of both cabal files and hpack files.
I am filing this as a good-first-issue kind of thing, not a request for you to write it necessarily.
Reacted by Simon Hengel, Hong Minhee (洪 民憙), August Johansson, Zoey and Satoshi Takimoto
In an ideal world we would derive the schema and documentation from the same representation that we use for parsing. However, I'm not sure how practical this is right now.
I think this are actionable items:
Make a proof of concept for some smallish type; trying to derive the schema and documentation, in addition to the JSON instances. We don't care much about the exact type representation, as we have dedicated types that are only used for parsing. So it's ok to clutter these types with TypeLits all over the place.
If somebody wants to produce the schema in a more direct / manual fashion then that will be great too. This may yield results faster and could serve as a reference / golden test for a generic implementation, if we ever get to it.
Allows for field aliases (and by extension for renaming of fields in a backwards compatible way)
Not sure what's the situation here with other implementations + ideally, we would want to derive the reference (those tables in our current README) from that same representation.
@NorfairKing just in case, are these things within the scope of Autodocodec? Where are the tests located in the source tree btw? I just took a look but only saw a Doctest driver.
The warnings are in scope but not implemented.
The field aliases are already supported.
Deriving a json schema is the exact use-case of autodocodec.
Even better would be a human-readible syntax-highlighted schema that you get with autodocodec-yaml.
Where are the tests located in the source tree btw? I just took a look but only saw a Doctest driver.
@NorfairKing looking at the example from the README, links to the generated output (schema, sample JSON, etc) could be a nice addition. I am not on a computer right now, so I can't try. And even when I'm back at the computer, I am swamped with a plethora of other things.
Questions:
From what I understand, the HasCodec instance is used to derive all the other instances. The example still derives a Generic instance; for what is that instance used/necessary? Or is that Generic instance unused?
Is there a generic implementation for HasCodec, or some other "generic way" to utilize this library?
If there is no generic implementation, have you tried implementing this in a fully generic way? If yes, did you hit any road blocks?
@NorfairKing looking at the example from the README, links to the generated output (schema, sample JSON, etc) could be a nice addition. I am not on a computer right now, so I can't try. And even when I'm back at the computer, I am swamped with a plethora of other things.
The example still derives a Generic instance; for what is that instance used/necessary? Or is that Generic instance unused?
It's not necessary, that's used for another part of the tests.
Is there a generic implementation for HasCodec, or some other "generic way" to utilize this library?
No, and that's the point. You're forced to document every field in the schema (or deliberately circumvent that)..
If there is no generic implementation, have you tried implementing this in a fully generic way? If yes, did you hit any road blocks?
Yes I have. The roadblock is that it's a bad idea because the entire point is to document your implementation, which you are circumventing using a generic implementation.
@NorfairKing looking at the example from the README, links to the generated output (schema, sample JSON, etc) could be a nice addition. I am not on a computer right now, so I can't try. And even when I'm back at the computer, I am swamped with a plethora of other things.
I looked at that, it was just not immediately clear to me which exact files correspond to the example from the README. Not that important, though. Once you understand what the example is doing it's easy to imagine how the schema will look like.
The example still derives a Generic instance; for what is that instance used/necessary? Or is that Generic instance unused?
It's not necessary, that's used for another part of the tests.
Is there a generic implementation for HasCodec, or some other "generic way" to utilize this library?
No, and that's the point. You're forced to document every field in the schema (or deliberately circumvent that)..
Ok, I guess that makes sense. For Hpack specifically we could annotate the fields with type literals (as we already do for aliases) and generically derive the documentation from that. But I think that's only really an option if you use those types for parsing only, and nothing much else.
I like how the encoder encapsulates everything in a single value, btw.
For completeness, I think conceptually this is what we would want:
{-# LANGUAGE TypeFamilies #-}
{-# LANGUAGE DataKinds #-}
modulePersonwhereimportData.CoerceimportGHC.GenericsimportGHC.TypeLitsnewtypeAnnotatedField (documentation::Symbol) a=AnnotatedFieldadataAnnotateddataParsedtypefamilyFieldrepresentation (documentation::Symbol) atypeinstanceFieldAnnotateddocumentationa=AnnotatedFielddocumentationatypeinstanceFieldParseddocumentationa=adataPerson_representation=Person{personName::Fieldrepresentation"name of person"String
, personAge::Fieldrepresentation"age of person"Int}derivingGenerictypePerson=Person_ParsedtypeAnnotatedPerson=Person_AnnotatedderivinginstanceShowPersonderivinginstanceEqPersonparseAnnotatedPerson::String->AnnotatedPerson
parseAnnotatedPerson = genericParse
where
genericParse =undefined-- add generic implementation hereparse::String->Person
parse =undefined-- coerce . parseAnnotatedPersonschema::AnnotatedPerson->String
schema = genericSchema
where
genericSchema =undefined-- add generic implementation here
Note that:
For simplicity, the given example code does not use type classes (ToSchema, ...). We would probably still want them.
Contrary to what I said earlier, I think this approach can be used with any types, not only with "types for parsing only", as long as you are willing to parameterize your types.
Technically, you could provide generic implementations that work for both annotated and unannotated types. If, however, you want to enforce documentation, then it's possible to constrain the generic implementation so that it only works with annotated types.
Person and AnnotatedPerson are representationally equivalent. Hence it should be possible to coerce one into the other. However, I couldn't convince GHC to do so. So we would need to do one of:
Somehow convince GHC to coerce.
Use unsafeCoerce, provided that we can encapsulate it in a way that provides a safe API to the user.
Implement the unwrapping generically.
For Hpack we would need to extend AnnotatedField with additional information (cabal name, aliases, deprecation..).
Activity
sol commentedon Sep 18, 2022
Sounds good. 👍
In an ideal world we would derive the schema and documentation from the same representation that we use for parsing. However, I'm not sure how practical this is right now.
I think this are actionable items:
tfausak commentedon Sep 18, 2022
I haven't used it before, but you may be able to use Autodocodec for this.
sol commentedon Sep 18, 2022
We currently have our own ad-hoc generic machinery in
Data.Aeson.Config.FromVa 8000 luethat:Not sure what's the situation here with other implementations + ideally, we would want to derive the reference (those tables in our current README) from that same representation.
@NorfairKing just in case, are these things within the scope of Autodocodec? Where are the tests located in the source tree btw? I just took a look but only saw a Doctest driver.
NorfairKing commentedon Sep 18, 2022
The warnings are in scope but not implemented.
The field aliases are already supported.
Deriving a json schema is the exact use-case of autodocodec.
Even better would be a human-readible syntax-highlighted schema that you get with
autodocodec-yaml.The readme has a section on this: https://github.com/NorfairKing/autodocodec#tests
sol commentedon Sep 18, 2022
@NorfairKing looking at the example from the README, links to the generated output (schema, sample JSON, etc) could be a nice addition. I am not on a computer right now, so I can't try. And even when I'm back at the computer, I am swamped with a plethora of other things.
Questions:
HasCodecinstance is used to derive all the other instances. The example still derives aGenericinstance; for what is that instance used/necessary? Or is thatGenericinstance unused?HasCodec, or some other "generic way" to utilize this library?NorfairKing commentedon Sep 18, 2022
Those are here: https://github.com/NorfairKing/autodocodec/tree/master/autodocodec-api-usage/test_resources
It's not necessary, that's used for another part of the tests.
No, and that's the point. You're forced to document every field in the schema (or deliberately circumvent that)..
Yes I have. The roadblock is that it's a bad idea because the entire point is to document your implementation, which you are circumventing using a generic implementation.
sol commentedon Sep 18, 2022
I looked at that, it was just not immediately clear to me which exact files correspond to the example from the README. Not that important, though. Once you understand what the example is doing it's easy to imagine how the schema will look like.
Ok, I guess that makes sense. For Hpack specifically we could annotate the fields with type literals (as we already do for aliases) and generically derive the documentation from that. But I think that's only really an option if you use those types for parsing only, and nothing much else.
I like how the encoder encapsulates everything in a single value, btw.
sol commentedon Sep 27, 2022
For completeness, I think conceptually this is what we would want:
{-# LANGUAGE TypeFamilies #-} {-# LANGUAGE DataKinds #-} module Person where import Data.Coerce import GHC.Generics import GHC.TypeLits newtype AnnotatedField (documentation :: Symbol) a = AnnotatedField a data Annotated data Parsed type family Field representation (documentation :: Symbol) a type instance Field Annotated documentation a = AnnotatedField documentation a type instance Field Parsed documentation a = a data Person_ representation = Person { personName :: Field representation "name of person" String , personAge :: Field representation "age of person" Int } deriving Generic type Person = Person_ Parsed type AnnotatedPerson = Person_ Annotated deriving instance Show Person deriving instance Eq Person parseAnnotatedPerson :: String -> AnnotatedPerson parseAnnotatedPerson = genericParse where genericParse = undefined -- add generic implementation here parse :: String -> Person parse = undefined -- coerce . parseAnnotatedPerson schema :: AnnotatedPerson -> String schema = genericSchema where genericSchema = undefined -- add generic implementation hereNote that:
ToSchema, ...). We would probably still want them.PersonandAnnotatedPersonare representationally equivalent. Hence it should be possible tocoerceone into the other. However, I couldn't convince GHC to do so. So we would need to do one of:coerce.unsafeCoerce, provided that we can encapsulate it in a way that provides a safe API to the user.For Hpack we would need to extend
AnnotatedFieldwith additional information (cabal name, aliases, deprecation..).lf- commentedon Sep 29, 2022
I speculate unconfidently that you might need a usage of the RoleAnnotations extension to make that coercion work: https://gitlab.haskell.org/ghc/ghc/-/wikis/roles