Re: [MirageOS-devel] Camlp4 Stack_overflow

[Anil Madhavapeddy <anil@xxxxxxxxxx>]

Just an update on this: I've added code to the Mirage CLI to exclude
<static_*> files (generated by crunch).  It'll be in Mirage 2.1.1:

https://github.com/mirage/mirage/pull/338

-anil

> On 8 May 2014, at 10:56, Andrà NÃss <andre.naess@xxxxxxxxx> wrote:
> 
> It looked like the snippet of code I included in the first post,
> except with 25,000 rows.
> 
> The _tags file solved it though. I excluded the file like this:
> <Pb.*>:-syntax(camlp4o)
> 
> With this in place I successfully compiled 500K entries in 1m30s
> yielding a 33MB .xen file.
> 
> I can certainly post an issue on the camlp4 github site and attach an
> example file if you think this should reasonably work. I wouldn't
> think this represent a common usecase ;)
> 
> On Thu, May 8, 2014 at 10:33 AM, Thomas Gazagnaire
> <thomas@xxxxxxxxxxxxxx> wrote:
>> do you have an example of what the generated source file looks like ?
>> 
>> Would be interesting to report the bug upstream: 
>> https://github.com/ocaml/camlp4/issues
>> 
>> Thomas
>> 
>> On 7 May 2014, at 23:38, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:
>> 
>>> To address the camlp4 problem, you can use the _tags file to mark just
>>> the one source file as not passing through camlp4 (via a '-' somewhere
>>> to remove the tag -- will need to look up the details in the manual).
>>> 
>>> The easiest way to solve your problem would be just to have a crunched
>>> file containing the information in CSV or S-expression format, and then
>>> reading from it at application startup time to initialize your array.
>>> 
>>> The 'crunch' mode would compile all this into a standalone kernel, or
>>> in Unix mode would be a filesystem passthrough for easier development.
>>> Any reason you want it to be a 'hand written' OCaml file?
>>> 
>>> -anil
>>> 
>>> On 7 May 2014, at 14:06, Andrà NÃss <andre.naess@xxxxxxxxx> wrote:
>>> 
>>>> I've rewritten the generated code so that it creates an array and then
>>>> assigns to indexes of the array, batching the assignments using
>>>> multiple let () =
>>>> 
>>>> Compilation speed is a problem, and it still falls over at 50K entries.
>>>> 
>>>> I suppose I could somehow write the data to an image and read it back
>>>> in at application startup, but I think I actually like my current
>>>> approach. All of the work is done at compilation time. The rows are
>>>> written in sorted order so a simple binary search can be used. The
>>>> resulting application is entirely single-purposed.
>>>> 
>>>> Any ideas for other ways to do this? Bypass the compiler and "compile"
>>>> the data myself? Does there exist tools to do something like that?
>>>> 
>>>> On Wed, May 7, 2014 at 1:40 PM, Andrà NÃss <andre.naess@xxxxxxxxx> wrote:
>>>>> Hi
>>>>> 
>>>>> I'm trying something that might be a bit silly but it's just intended
>>>>> as an internal demo of Mirage to my team. I'm writing a tiny HTTP
>>>>> based application whose only purpose is to allow you to search a phone
>>>>> book.
>>>>> 
>>>>> I've based my approach on the approach you've used for static websites
>>>>> using crunch where you compile the web pages into the executable. In
>>>>> my case that means I take the list of name and number pairs and stuff
>>>>> them into an array. So part of my config.ml file emits a single Pb.ml
>>>>> file which contains an array declaration like this:
>>>>> 
>>>>> let db = [|("Aaron Jaksic",1571268);
>>>>> ("Aaron Mcdivitt",1454629);
>>>>> ("Aaron Reuben",1765247);
>>>>> ("Aaron Sardinha",1224806);
>>>>> ("Abbey Gowens",1267083);
>>>>> ("Abbey Hathcox",1309490);
>>>>> ...
>>>>> 
>>>>> As the title say I'm running into problems compiling this when I
>>>>> create a large phone book, the error being:
>>>>> 
>>>>> Camlp4: Uncaught exception: Stack_overflow
>>>>> 
>>>>> This happens with 25K entries, it works with 10K entries.
>>>>> 
>>>>> Is there some way I can prevent Camlp4 from being run on the Pb.ml
>>>>> file? Or will I just have to generate a file that builds the array at
>>>>> runtime somehow?
>>>>> 
>>>>> AndrÃ
>>>> 
>>>> _______________________________________________
>>>> MirageOS-devel mailing list
>>>> MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
>>>> http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel
>>> 
>>> 
>>> _______________________________________________
>>> MirageOS-devel mailing list
>>> MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
>>> http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel
>> 
> 
> _______________________________________________
> MirageOS-devel mailing list
> MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
> http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel


_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel