Arvados

Building R documentation isn't working:

2018-03-23_17:29:17.62809 > pkgdown::build_site()
2018-03-23_17:29:17.62811 Error in switch(fun, `<-` = paste0(fun_name(expr[[2]]), "<-"), S3method = ,  : 
2018-03-23_17:29:17.62811   EXPR must be a length 1 vector
2018-03-23_17:29:17.62811 Calls: <Anonymous> ... <Anonymous> -> .Call -> .f -> <Anonymous> -> .Call -> .f
2018-03-23_17:29:17.62811 Execution halted

Unfortunately the error is pretty cryptic.

Could you document how to run autoGenAPI.R ?

It looks like getProject(), listProjects() and so forth are obsoleted by the autogenerated API (that's good). However, it looks like instead of turning them into stubs that call groups.get() and groups.list(), they are gone entirely, but still documented in README.Rmd.

Also, README.Rmd is lacking any documentation on using the new autogenerated API.

• The methods "index", "show" and "destroy" are aliases for the preferred names "list", "get" and "delete". We should probably fix the discovery document so it doesn't publish redundant methods, but in the mean time I suggest filtering out "index", "show" and "destroy".

• For development and testing, we have an environment variable ARVADOS_API_HOST_INSECURE which specifies that the HTTP client should not try to validate the server's TLS certificate (this allows us to use self-signed certificates generated on the fly by the test suite). Please see if you can support this flag in the R SDK, it is probably a configuration option to httr or curl.

I haven't tried every function, but spot-checking various autogenerated list(), get(), delete(), create() and update() everything works well so far!

Fuad: 13076-r-autogen-api back to you with above comments.

Methods projects.get, projects.create, projects.delete, projects.update and projects.list are now defined on Arvados class (they wrap groups.list ... by adding "group_class" attribute where necessary).

Methods that end on "index", "show" and "destroy" are removed from auto-generated Arvados class (reduced file by more than 2000 lines).

When environment variable ARVADOS_API_HOST_INSECURE is set to TRUE all http request will contain ssl_verifypeer options field set to FALSE.

README.Rmd is updated to reflect changes made in Arvados class.

autoGenAPI.R file contains one public function called generateAPI which doesn't receive arguments. This function,
when called, will generate Arvados.R file which contains main Arvados class. My idea was to generate Arvados.R file by calling generateAPI and put that file into ./R folder of project. Then we need to call document function (defined inside devtools package) to generate documentation. After that we can build package (without autoGenAPI.R file). generateAPI can be modified to take discovery document and/or file path, where to put Arvados.R file, as arguments.

Currently generateAPI is exported as public function (since I don't know what exactly to do with it and I need to use it) and can be called directly.

Most of the documentation that correspond to method arguments is missing from discovery document. So executing, for example, ?jobs.list will not give a very clear picture what method arguments represent. Also documentation for methods is preformatted. For example, documentation for collections.list looks like this:

List Collections.

The list method returns a
                   <a href="/api/resources.html">resource list</a> of
                   matching Collections. For example:

                   {
                    "kind":"arvados#collectionList",
                    "etag":"",
                    "self_link":"",
                    "next_page_token":"",
                    "next_link":"",
                    "items":[
                       ...
                    ],
                    "items_available":745,
                    "_profile":{
                     "request_time":0.157236317
                    }
                    

Not sure If it's possible to can put this in R documentation as description.

Most of the class descriptions (example discoverDocument.schemas.Collection.properties) contain duplicated uuid field.

Currently the main blocker for merge is the online HTML documentation generation. pkgdown is failing as noted in https://dev.arvados.org/issues/13076#note-6 so either we need to determine why pkgdown is failing (and fix it) or stop using pkgdown and do something else. I would be fine with only processing the contents of README.Rmd and not worrying about method-level documentation. I think processing Rmd files requires using "knitR"

By the way, the current documentation generated is implemented in arvados/doc/Rakefile.

Peter Amstutz wrote:

By the way, the current documentation generated is implemented in arvados/doc/Rakefile.

I added createDoc.R script which can be run by $Rscript createDoc.R README.Rmd README.html.
I requires knitr and markdown packages. Could you update Rakefile?

Alright, I spent some time with it and now it autogenerates mostly reasonable online documentation including both the README and function/method documentation. I pushed to the 13076-r-autogen-api branch.

Please go ahead and merge to master.