{{ message }}
Repository navigation
Building against precompiled dependencies #2477
Description
You can see an example of this in this Nix setup I'm experimenting with. The problem is that psc doesn't provide any means for depending on code you've already compiled. If you want to use precompiled code, you have to copy it into the output directory, and include the sources for that code in the list of sources for psc. This is at best clumsy, and is prone to issues with timestamps changing on copy. It'd be nice if there were some intermediate representation (corefn?) that preserved the necessary information for compiling against.
Activity
paf31 commentedon Dec 12, 2016
The data format you want is the
externs.jsonformat, but there is no command line API for building against a set of externs files right now. We could add one though. The closest thing right now ispsc-ide's rebuild command.ElvishJerricco commentedon Dec 12, 2016
Ah cool. Yea having a CLI for building against
externs.jsonsounds like the perfect solution. Are you saying thatpsc-ideis already doing this sort of thing? Meaning that would be a good place to look if I wanted to try implementing this?paf31 commentedon Dec 12, 2016
Yes,
psc-ide, PSCi and Try PureScript all use this method. It's handled by therebuildfunction inMake.hs.ElvishJerricco commentedon Dec 12, 2016
Cool. I'll take a look at it and let you know if I have the time to work on a PR for this.
paf31 commentedon Dec 12, 2016
Great, thanks! If you do, let's make sure to hash out a basic plan for the command line interface here so that others can review it too.
ElvishJerricco commentedon Dec 12, 2016
Sure. My first thoughts are to make it similar to the classical C compiler flags for external libraries.
psc -L/path/to/somelib/output/ $mysourcesThis would tell
pscthat it can find theexterns.jsonfiles for modules insomelibat/path/to/somelib/output/ModuleName/externs.json.ElvishJerricco commentedon Dec 17, 2016
So I've been looking at this today. I think there needs to be more than just a flag for
psc. Withghcin Nix, you can usehaskellPackages.ghcWithPackages (self: [ self.async self.transformers ... ])to get aghcthat automatically compiles against whatever packages you need. So when you enter a nix-shell with such aghc, you can useghc Main.hs, orghci, or evencabal configure, and they all just automatically include the packages you specified from Nix. I'm not sure, but I believe this is accomplished by replacingghcwith a script that callsghcwith the correct arguments (and the same goes forghci).To get the same behavior with the
psctoolchain, there would have to be a flag not just inpsc, but also inpsci,psc-bundle, etc. etc.. Does this sound too broad-scoped?ElvishJerricco commentedon Dec 23, 2016
To do what I suggested in that last comment, a number of things would have to be refactored to work with
ExternsFileinstead ofModule, since there's no way to recover aModulefromExternsFileAFAIK. So for now I'm just going to see if I can getmaketo build against a list of(ModuleName, ExternsFile), and pass an empty list everywhere butpsc.paf31 commentedon Dec 23, 2016
@ElvishJerricco You shouldn't need full modules for dependencies to just compile a single module. You just need an
Environmentwhich contains their type definitions. TheExternsFilestructures can be used to create anEnvironment.ElvishJerricco commentedon Dec 23, 2016
Ah, I think I see my mistake. I thought this field was the only way the
pscigot a hold of any definitions. I see now that this field provides anEnvironment. Does completion work with definitions in theEnvironment? Looking throughCompletion.hs, it seems to useModulea lot.paf31 commentedon Dec 23, 2016
I would expect it to use the
Environment, but I'm not sure to be honest.ElvishJerricco commentedon Dec 23, 2016
At the very least, I'd expect it to be possible with the current
Environmentformat. So worst case scenario, completion just needs a bit of a refactor.ElvishJerricco commentedon Jul 14, 2017
In case it's not clear by now, I will not be completing this feature myself, should anyone else want to give it a shot.
denibertovic commentedon Sep 10, 2019
I was wondering what the current state of this issue is. @paf31 I know I probably shouldn't ping you but perhaps you would know who to ask instead?
hdgarrood commentedon Sep 10, 2019
I think work on this issue specifically should wait until there has been progress on #3493.
joneshf commentedon Mar 22, 2020
@hdgarrood While I understand that #3493 and this issue are related, and I believe I understand what you explained here, it's not clear why this work cannot happen until #3493 gets moving. Can you explain more what is blocking work from progressing here?
I'd like to make a PR to allow compiling against pre-compiled artifacts, but I don't want to do that if it would be rejected for doing things out of order. If it would, I'll refrain from making the PR.
hdgarrood commentedon Mar 22, 2020
We could start on this before #3493 gets moving, but the reason I’m a bit hesitant is basically that I think we are likely to end up with a better API if we address it via #3493 (as that would solve the need here and quite a few others besides). Various other tools in the ecosystem (pursuit, spago, even the way docs are handled inside the compiler) have had to come up with their own separate ways of dealing with packages, causing a few awkward corners, and this feels to me like it would take us further down that path. There’s also the issue that if we were to do this first and later came back and did #3493 (which I do want to do pretty soon, especially as we are starting to move away from Bower and to our own registry in purescript/registry) we would end up with more compiler API surface area, which means more potential places for bugs, more stuff that users need to get their head around, etc.
joneshf commentedon Mar 23, 2020
Thanks for explaining. That makes sense. That also makes it clearer that there's at least two different ideas spurred on by this issue:
The two sound similar, but the former is inter-package focused, while the latter is intra-package focused. The former is what the title of this issue and the initial comment are about; the latter is what the first response and most of the conversation changed to. As you mention, the former can be solved by making more progress on packages. The latter can be solved by providing a better UX for the behavior in
purswhich already exists. The former is what people using the language care more about; the latter is what people writing tooling care more about.Neither solution (or any other ideas) can truly solve the other issue. But either solution kind of gives a way to solve the other. It seems like we need a solution to both of these if we're going to introduce the idea of packages. Which is to say, I think the only way to have one solution is to not have the idea of both modules and packages.
I understand the concern for the
pursAPI, but there's also the very real problem of trying to use PureScript for non-trivial things right now. If you'd prefer there not to be a PR, I won't make one. But, if you're okay with it, I'll send one. I won't be offended either way you decide, but it's largely up to you.hdgarrood commentedon Mar 23, 2020
I’m not sure I agree with this distinction. Specifically I am not so certain that this proposal, if implemented, would provide a better interface to incremental compilation. It would of course give you more powerful interface, but, say, if in practice this leads to people getting errors at runtime because the API is too easy to use incorrectly, that’s maybe not such a good step. The behaviour that already exists has a fairly clear boundary between what is the responsibility of the user and what is the responsibility of the compiler: the user just has to supply all the source files and not fiddle with the output directory, and the compiler takes care of everything else.
I also don’t follow your reasoning for why we shouldn’t have both modules and packages. Partially it’s that I disagree with the premises (see above), but also it’s not clear why we need a solution to both if we are going to introduce the concept of packages.
I’m not saying I want to rule out doing this entirely, but I would prefer not to have a PR at this stage.
joneshf commentedon Mar 24, 2020
Fair enough.
Start working on a batch compiler
Support pre-compiled artifacts
joneshf commentedon Mar 30, 2021
If anybody else is still blocked by this issue and would like to try a drop-in replacement for
purs compilethat builds against pre-compiled dependencies, I'm working on apurs-compilebinary that does just that.Caveats:
npm install @joneshf/purs-compileor something. Will come later.More than happy to field questions over in that repo so as not to clutter up this Issue.