Activity

1. bnoordhuis commented on Nov 3, 2014

   

   bnoordhuis

   on Nov 3, 2014

   Member

   More actions

   writing dynamic tests for the Node.js core itself that fuzzes the native methods and detects if errors are correctly thrown in case of invalid parameters

   That's not a bad idea! Although the test suite should cover the API surface pretty well, more validation won't hurt.

   I do see a practical issue though: there are a fair number of API methods that allow more inputs than what is documented for backwards compatibility reasons. I'm not sure how many exactly but I imagine you could end up with a fuzzer that's a big pile of special cases.
2. Mickael-van-der-Beek commented on Nov 3, 2014

   

   Mickael-van-der-Beek

   on Nov 3, 2014

   Author

   More actions

   • Parameter overloading could be solved by having a list of possible parameter combinations.
     Here's an example in MongoDB's native Node.js driver docs:

   http://mongodb.github.io/node-mongodb-native/api-generated/collection.html#find

   • Backwards compatibility could be partly handled by versioning the specs in relation to the Core and then flagging the special cases as being linked to a particular past version.
3. darrenderidder commented on Nov 16, 2014

   

   darrenderidder

   on Nov 16, 2014

   More actions

   @Mickael-van-der-Beek - I like it. Another potential use case that a lot of people on the node group really wanted was an alphabetical API reference like Angular, et al, rather than just the man-page style. Last time I checked, the json docs didn't have quite enough consistency to build an API reference automatically.
4. Mickael-van-der-Beek

   added a commit that references this issue on Nov 18, 2014

   Refactoring of the module. The validator still needs improvement and …

   ...

   4b8d759
5. Mickael-van-der-Beek commented on Nov 18, 2014

   

   Mickael-van-der-Beek

   on Nov 18, 2014

   Author

   More actions

   @darrenderidder - Thanks! An alphabetical spec would definitely be a cool improvement!

   Currently the JSON and HTML API specs are generated with a non-NPM-published module called node-doc-generator developed by @isaacs.

   I feel that although it's very convenient, I'd prefer to do it the other way around.
   e.g: JSON API specs generate HTML and maybe also let description strings be markdown that is than converted to HTML.

   Here is an example of the schema syntax I have in mind:

   {
       name: 'fs',
       type: 'module',
       methods: [{
           name: 'readFileSync',
           type: 'method',
           signatures: [{
               name: 'filename',
               type: 'String',
               format: [
                   'notEmpty',
                   'isPath'
               ]
           }, {
               name: 'encoding',
               optional: true,
               default: 'utf8',
               type: 'String',
               format: [
                   'notEmpty',
                   'isEncoding'
               ],
               deprecated: '0.10.25'
           }]
       }]
   }
   

   The type key, when not used in a signature description, can take three possible values: class, module and method.

   The deprecated key is an optional flag that displays a visual warning to the user reading through the specs to underline that it is not recommended to specify this parameter.

   I have already written and tested (86% coverage) a spec validator module in a new repo I created called Node Cerberus: Mickael-van-der-Beek/node-cerberus@864f93a.

   It supports type and format validation like above but also various options that can be configured at the module level or spec level:

   • typeStrict (default=true): validation should fail in case the type doesn't match the spec
   • formatStrict (default=true): validation should fail in case the format doesn't match the spec
   • existenceStrict (default=true): validation should fail in case a parameter is not specified
     (like a global { optional: true })
   • nullAsExistence (default=false): null is considered as an existing (as in not missing) value
   • undefinedAsExistence (default=false): undefined is considered as an existing (as in not missing) value

   If you want more details about the spec format for nested Objects / Arrays or wildcard Array element validation, PM me or look at the tests for node-cerberus.
   I'm sorry, I didn't have the time to write documentation yet.

   The fuzzer module, in the same repo is still a work in progress but I already created a directory containing a list of "special" values for the Node.js + JavaScript native type, that will be used as payloads during fuzzing tests. Link below:

   Mickael-van-der-Beek/node-cerberus/src/fuzzer/payloads

   What do you guys think ?

   PS: One case I'm currently not handling in my validation spec format is linked parameters. e.g:

   function MyMethod (fileData, fileEncoding)

   where fileData is expected to be a String encoded using the fileEncoding encoding.

   One possible solution for this:

   signatures: [{
       name: 'fileData',
       type: 'String',
       format: [
           {
               name: 'isEncoding',
               args: ['ref:fileEncoding']
           },
           'notEmpty'
       ]
   }, {
       name: 'fileEncoding',
       type: 'String'
   }]
   

   Which means that a format could be specified as a String (representing a validation method name) or an Object containing a name key for the method name and an Array of args that will be applied to the validator.

   The 'ref:fileEncoding' argument here would be the fileEncoding variable value at validation time.

   It's a bit of a hack so I'm definitely open other ideas !