Re: ocamldoc generation and packed files

[Anil Madhavapeddy <anil@xxxxxxxxxx>]

Oh, if it's easy that would be pretty useful!  I could also modify ocaml-annot 
to handle the packed versions (I need to hack it to understand _build 
directories anyway at some point).

I'd quite like to have a 'blessed' vim and emacs bundle that works with 
Mirage+extensions by the time the tutorial comes around, as right now dealing 
with the syntax extensions is quite problematic.  I think between us we have a 
reasonable mix of vim/emacs users (Raphael and I use vim, and Balraj and you 
use emacs at least!).

Anil

On 19 Aug 2011, at 12:27, Thomas Gazagnaire wrote:

> Well, actually it's seems quite easy to do, as the packed .annot file 
> contains already the right location information. I can try to work out on a 
> quick-and-dirty tool to fix your issue :-)
> 
> 
> On Aug 19, 2011, at 1:21 PM, Thomas Gazagnaire wrote:
> 
>> I don't have concrete available solution for the .annot files. The two I can 
>> see are :
>> 
>> * releasing the .ml -> .annot part of the compiler as a stand-alone tool and 
>> use it to generate individual .annot files (I guess that's not very 
>> different from running ocamlc -annot on every file)
>> * having a tool to split out a packed .annot file into individual .annot 
>> files. I guess it's not so difficult to do (just need to parse the directive 
>> lines in the packed source file + its annot file and do some comparison), 
>> but we don't have it yet.
>> 
>> Thomas
>> 
>> On Aug 19, 2011, at 11:58 AM, Anil Madhavapeddy wrote:
>> 
>>> As a followup to this, I've integrated ocaml-ocp into my tree and it's 
>>> working really well. Thanks for releasing this Thomas!  There was an 
>>> off-by-one in the cpp directives for line numbering (fixed in our tree).
>>> 
>>> It would be very useful to have some way for .annot files to work without 
>>> compiling the individual sub-files; any thoughts on how this might work?
>>> 
>>> Anil
>>> 
>>> On 13 Aug 2011, at 20:39, Anil Madhavapeddy wrote:
>>> 
>>>> Sure, go for it if you get a chance.
>>>> 
>>>> The current packing "works", but the doc target cannot be built at the 
>>>> same time as the normal code (because it causes the source pack to leave 
>>>> .ml files in lib/std/foo.ml, which then causes the lib/std/foo.cm{xo} to 
>>>> be rebuilt instead of the -packed copied .cmx to be used).
>>>> 
>>>> Basically, its a bit of a mess :-)
>>>> 
>>>> As a sneak preview, check out http://github.com/avsm/mirage-browser.git 
>>>> (on index.html). This is a *very* rough jQuery-based module browser of the 
>>>> Mirage OCamldoc that I'd like to have live on the website before the CUFP 
>>>> tutorial.  The live search is pretty nice, and I'll flesh out the 
>>>> rendering over the next few days.
>>>> 
>>>> This tool is also quite standalone as it uses the output of 
>>>> odoc_json/ocamldoc, and so could be applied to the standard distribution 
>>>> and also to the Citrix xapi-libs (if you're interested, David). 
>>>> 
>>>> Killer feature: the js_of_ocaml interactive top-level should work in 
>>>> here... :-)
>>>> 
>>>> Anil
>>>> 
>>>> On 13 Aug 2011, at 17:50, Thomas Gazagnaire wrote:
>>>> 
>>>>> No ocamlbuild integration yet but would be definitely useful to add. I
>>>>> can have a look at integrating it to mirage build next week if it's
>>>>> useful.
>>>>> 
>>>>> --
>>>>> Thomas
>>>>> 
>>>>> 2011/8/13 Anil Madhavapeddy <anil@xxxxxxxxxx>:
>>>>>> Thanks Thomas, that looks very useful and definitely better than the 
>>>>>> script in tree at the moment.
>>>>>> 
>>>>>> One question: ocp-pack also requires that the list of files be passed in 
>>>>>> dependency order, or else the resulting pack file will not compile.  
>>>>>> Have you tried to integrate it as an ocamlbuild rule, so that it can 
>>>>>> automatically sort the input modules before generates the packed file / 
>>>>>> functor?
>>>>>> 
>>>>>> Anil
>>>>>> 
>>>>>> On 12 Aug 2011, at 20:01, Thomas Gazagnaire wrote:
>>>>>> 
>>>>>>> It will be anounced quite shortly on the ocaml mailing list, so I can
>>>>>>> give you the link to ocp-pack :
>>>>>>> 
>>>>>>> http://www.ocamlpro.com/code/2011-08-10-ocaml-pack-functors.html
>>>>>>> http://www.ocamlpro.com/files/ocp-pack-1.0.0.tar.gz
>>>>>>> 
>>>>>>> It should help you to pack modules easily and have a working doc.
>>>>>>> 
>>>>>>> --
>>>>>>> Thomas
>>>>>>> 
>>>>>>> 2011/7/28 Anil Madhavapeddy <anil@xxxxxxxxxx>:
>>>>>>>> That tool will be very useful! I've committed the 'make doc' target. 
>>>>>>>> The output is still pretty dirty as we don't use the documentation 
>>>>>>>> tags properly, but I'll go through adding .mli files and adding proper 
>>>>>>>> documentation on the more stable interfaces.
>>>>>>>> 
>>>>>>>> It would be quite nice to eventually have a single documentation 
>>>>>>>> output for all the backends, with an addition section saying 'only 
>>>>>>>> present in Xen' or 'only present in Node'. I think that should be 
>>>>>>>> possible by parsing the ocamldoc dump outputs, but something for the 
>>>>>>>> future!
>>>>>>>> 
>>>>>>>> Anil
>>>>>>>> 
>>>>>>>> On 27 Jul 2011, at 16:56, Thomas Gazagnaire wrote:
>>>>>>>> 
>>>>>>>>> We have as well a program which pack together ML files (and is able 
>>>>>>>>> to functorize packs as well...) it is not released yet, but I guess 
>>>>>>>>> we can open-source it shortly.
>>>>>>>>> 
>>>>>>>>> But I think overriding the default rule for ocamldoc+pack in 
>>>>>>>>> ocamlbuild is sufficient for now on so you should push your patch :-)
>>>>>>>>> 
>>>>>>>>> Thomas
>>>>>>>>> 
>>>>>>>>> On Jul 27, 2011, at 5:36 PM, Anil Madhavapeddy wrote:
>>>>>>>>> 
>>>>>>>>>> I'd really like to generate ocamldoc (HTML/PDF) of all the various 
>>>>>>>>>> libraries so that it's easier to learn Mirage (and support editor 
>>>>>>>>>> auto-completion, etc).
>>>>>>>>>> 
>>>>>>>>>> The big problem is that ocamldoc doesn't support packed modules, and 
>>>>>>>>>> we use packing quite extensively (in Net, Http, Block, etc).
>>>>>>>>>> 
>>>>>>>>>> So I've hacked up an ocamlbuild target that concats together the 
>>>>>>>>>> *source* ML files from an .mlpack and uses that to generate the 
>>>>>>>>>> ocamldoc for the standard library, with one set of HTML files 
>>>>>>>>>> generated per backend (Xen, Net-Direct, Net-Socket, and so on).
>>>>>>>>>> 
>>>>>>>>>> The issue is that these rules are a little grim: everywhere where we 
>>>>>>>>>> have a .mlpack file at the moment, we need to override that rule to 
>>>>>>>>>> generate a concatenated ML file that is used for ocamldoc (but not 
>>>>>>>>>> for actual compilation, since line numbers get lost since those 
>>>>>>>>>> aren't preserved when converting from ML files into a single big 
>>>>>>>>>> one).
>>>>>>>>>> 
>>>>>>>>>> Does anyone have a better solution for ocamldoc and packed files? 
>>>>>>>>>> All of the grimness here is hidden away in the ocamlbuild rules, so 
>>>>>>>>>> I'm inclined to just commit this patch, and perhaps see about adding 
>>>>>>>>>> -pack support into ocamldoc at a later stage (there's an open bug in 
>>>>>>>>>> Mantis somewhere).
>>>>>>>>>> 
>>>>>>>>>> Dave, do you use pack in XAPI, and/or ocamldoc?  I wonder if 
>>>>>>>>>> everyone else (like Core) also have their own swanky 'cat ML files 
>>>>>>>>>> into one' script too...
>>>>>>>>>> 
>>>>>>>>>> -anil
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>> 
> 
>