Still Alive/Lost Momentum? #1571
Comments
|
We are still here. Unfortunately COVID has not meant free time. Quite the opposite. We chat in the Gitter channel so you can see us interacting. Simple, direct answer. We are stuck needing to update our code because it no longer works on newer MacOS which is a common platform. There is this branch: https://github.com/remacs/remacs/tree/reset-to-emacs-27 where we are working to catch up to recent Emacs where we know fixes have been made. |
|
For the most part, the stuff that's left to do is hard. The garbage collector, the bytecode interpreter, anything like that. If we were all getting paid to work on it I'm sure we could figure it out, but as things are now, it's quite a challenge. It's important to keep some perspective though. If we look at Remacs as a serious attempt to replace the C core of Emacs with Rust, then we can safely call it a failure, because that isn't going to happen any time soon, maybe ever. BUT, if we look at Remacs as a proof-of-concept to show that it would be possible to do such a replacement, then I would call it a wild success. |
|
Although I'm glad there have been more upstream changes than I expected, I think it would have been possible if there wouldn't be that much new code we have to deal with. It's work that doesn't really pay off. Maybe we can see this as a decision of the emacs community to stick with things how they are. It's not realistic that such an ambitious project can be handled by a few people since we have to update remacs for code by many more people that lands in upstream. |
|
Let's keep this open for anybody who is interested. |
|
Perhaps it is time to revisit the automatic conversion idea again? It may not be idiomatic but a pure Rust core is a lot easier to refactor than the mess that is C. |
|
I think it's not that easy to convert the complete code base. |
|
Yes, but we could start it from scratch instead, focus on the UI and functionality instead of implementation itself. We could make a list of priorities and code them one by one. This is how Vi-inspired editors written in Rust are being developed. Well, the first steps are going to be a pain though. But we don't have to keep up with GNU-Emacs this way. Rust has some native support for functional programming, doesn't it? Also, since GNU Emacs seems a bit too complicated, we can start with this instead. |
|
I have absolutely no idea how complicated this would be for emacs, but I think this would be far more work than for vim. I guess you are right that there are less conflicts we have to deal with. Did you already take a look at the display engine related code ? It looks quite complex to me =) |
|
Here is the deep technical answer. This port was going to hit a wall at some point. Emacs uses some C tricks to make its lisp compilation work and to load faster by pre-configuring some of the lisp code. Rust's memory model and other bits make this a non-trivial change. Similarly, the allocator and lisp garbage collector use knowledge of C's stack and heap. We can continue to chew around the edges of Emacs but at some point the low and mid level fruit will be gone. What is frustrating is the C code does not well define the elisp interpreter parts from the buffer or UI parts. This is something we can improve as we work on the Rust code. Unfortunately, we hit a different problem. We have a mix up in our port blocking usage on current MacOS. Which happens to be a significant portion of the user and developer base. It has proven hard to identify exactly what is wrong. I started a new branch to "reboot" the port to be on top of Emacs 27. This time instead of deleting code we are thinking about ifdef'ing it in the hope that will make later git merges easier while we still have C code under us. https://github.com/remacs/remacs/tree/reset-to-emacs-27 is the branch. Anyone can help out by working on adding the ifdef's and identifying shifts from the emacs26 we started with to the emacs27 we want to use. Once we are over this hurdle we can focus on the 'fun' parts of the port identified above. |
|
In case people haven't seen the gccemacs project, I thought I'd put a link to it here since perhaps it might have some relevance to this discussion of the future of this project? Also the author Andrea Corallo recently gave a talk on it which there is a link to on the project page. It sounds like the eventual goal for that project is to merge it into mainline emacs at some point. As a disclaimer, I'm not super knowledgeable about the implementation of the elisp interpreter, byte compiler, runtime or where remacs stands on oxidizing those parts of emacs since I've just been following these projects from the sidelines. My understanding is that if gccemacs (aka the native-comp branch) is merged, it would allow reducing the C footprint of emacs as performance critical functions previously written in C could be written in elisp and compiled. Perhaps even elisp itself could become self hosted. Given that the goal of this project is to port emacs' C to rust which includes the elisp bytecode interpreter and runtime, I was wondering if anyone has thought how elisp potentially being natively compiled and self hosted might impact the future of the remacs project? |
|
Building compilers and JIT interpreters in Rust is relatively straightforward with Cranelift. It would not be difficult to surpass native Emacs' performance given that a pure Rust core would be a lot more extensible and approachable, not to mention that it would also be amenable to more concurrency optimisations due to Rust's strong support for safe concurrency. A pure Elisp Emacs may be a lot easier to port but given the strong C legacy I do not think a self hosted Elisp Emacs would happen anytime soon. |
|
I'll remind everyone that you've chosen to permanently hard fork the emacs codebase by not implementing copyright assignment (which could just be a simple github tooling fix). This project deliberately excludes everyone that contributes to the existing Emacs codebase. Those are the people most able to actually port the hard parts to Rust. This project will never be upstreamed because of the simple lack of copyright assignment. |
This project doesn't intend to be upstreamed. It's not only about porting emacs to rust. We want emacs to be a community driven project. |
As a contributor to both Emacs and Remacs, I can assure you that this claim is totally wrong. Remacs does not exclude anyone, and the FSF copyright assignment does not prevent anyone from working on forks. |
Emacs is a community driven project? It has been for decades. Based on the words you've said here you, and the other contributors, have a deeper reason for a community and codebase hard fork. That's totally fine and part of free software. BUT you don't have a clear land page about this. https://github.com/remacs/remacs#why-a-fork You mention other projects but some of them at least are upstreamable because they did some minor paperwork (ie like the gccemacs author I'd guess). You should make it much more clear that you are permanently hard forking Emacs and the reasons why. Add just what you said to me "This project doesn't intend to be upstreamed. It's not only about porting emacs to rust. We want (r)emacs to be a community driven project." I think, for clarity and fairness to your users, you should add - "This is a permanent and intentional hard fork (that cannot be upstreamed) of the Emacs codebase because there has been no copyright assignment to the Free Software Foundation." On the landing page so users and potential contributors have at least some warning. |
|
From this discussion it sounds like a 1:1 replacement is difficult to impossible. Additionally it sounds like being eventually up-streamed is no longer (or never was?) an intention? If those two are true has a rewrite been considered instead? As in, instead of a 1:1 conversion have a new editor who's goal is to support the same elisp code as Emacs. A reason I'd be excited about such a project is largely the limitations of Emacs GUIs. A new implementation could make what @altaway talked about possible. I currently use the |
|
Going upstream could never happen. We all knew that coming in. The Rust ecosystem is not very FSF/GPL friendly. The bits of Emacs could end up in crates. The regex handling, ui, etc etc could all be pulled and teased apart. Under the Emacs umbrella for sure but in a very different way. This project has always been about exploring what is possible. Exploring the Rust options. Learning. Maybe some day talking to the FSF about whether C is really a realistic language. To be quite clear, we are not antagonistic to the FSF or their work. Emacs has demonstrated what could be done long before anyone else was trying. That legacy is important. But the project was started for reasons and those reasons have not changed. It would also mean replacing all of the existing Emacs devs or them learning the new language. That feels deeply me me me and not at all what open development should be about. I agree, GUI work is one of the things that interests me too. I personally am not interested in some web/javascript thing but others in the project are. This project needs people with interest to consistently contribute. Just like any other project. I have found it hard to contribute for various personal reasons. |
|
If this project needs to make anything more clear, please speak up.
I think this one line establishes the intent quite clear. We may well end up in some place very different. But some place that any person steeped in Emacs would recognize. Just like the Linux or FreeBSD of today is not all that similar to the Unix of the early 1990s. But in many ways still very connected. |
|
My personal pet peeve of emacs is that tools like this https://github.com/plexus/chemacs need to exist. I want to get to a point where I can |
|
Yeah, I really hope this project can pull through the hard times to a glorious future. I think most people today have no concept of the alternate history that is the Lisp Machines (I didn’t until a few years ago), and Emacs is the last descendant of that vision of computing. But, it’s become too weighed down with decades’ worth of kludges and compromises. The FSF and RMS have their reasons for being conservative about the codebase, which is fine, so an experimental fork makes sense as a way to test new ideas and plot ways past old problems. It’s never been about GNU Emacs vs. Remacs; it’s about making sure there’s an Emacs of any kind for future generations to enjoy, so the Lisp Machine dream, of computing of the people, by the people, for the people, shall not perish from the earth. |
It is a plausible future. We could reach a point where we re-implement everything. At the moment, being compatible with elisp has value -- we can use the existing infrastructure that exists. Doom, org, all of it. The problem with replacing too much of emacs is elisp and emacs are joined. We cannot replace/fix/whatever elisp without breaking significant parts of the community. If we say invented our own rlisp and based the editor around it, we would have to find a way to somehow run elisp code too. But the deep bits of elisp know about buffers, windows, and other things. Which is a lot of history to virtualize. Not saying it is impossible, but it adds to the lift. The good news is we can start by being a bug compatible Emacs implementation. Then we can decide how crazy to go. Or perhaps there are forks from Remacs that explore really out there ideas. I want there to be a base to work from. Let's worry about the future once we get there. |
|
I was thinking about replacing elisp with haskell-like syntax, with an interpreter written in Rust (like how elisp/s interpreter/core was written in C) and maybe call it "Rascal" or something like that. But there is some aversion to haskell in the emacs community and I'm not sure why. |
|
@altaway: That would be fun, though I can't really recommend it. There's over a million lines of elisp with all kinds of great functionality that really make Emacs what it is; losing that would give you an editor that isn't Emacs any more. |
|
I've noticed that the function (both lisp and primitive) descriptions have been modified in remacs. They've mostly been made more concise. Any specific reason? I thought the idea was just to rewrite the (originally C) core in Rust and nothing else. |
it's too complex 😕 😕 So.. This eliminates the need to write editor and front code, and solves problems with writing in Rust, modularization (VM, Editor, Front #1548) and performance. |
It also sounds like it would seriously cut into the functionality that elisp offers. For example, buffer management wouldn't necessarily be accessible from elisp at all, let alone work the same way. |
|
Hmm.. There's definitely a problem with practicality. It's just sample
|
|
Maybe it’s a good time to hard fork emacs and never look back? With all that is happening with RMS and FSF? If someone doesn’t know that I’m talking about |
|
Closing this in favor of https://github.com/emacs-ng/emacs-ng |
I'm absolutely no RMS fan, but this
isn't that much better than what he is doing. This kind of actions can make sense when you want to do something against amazon, microsoft etc. but not an old man, even if he has some weird opinions. I can understand that they don't want to work with him, neither do I. But then they should rather look for an alternative than bashing him on the internet. Sometimes it seems to me that people who criticize others for not being tolerant, are even more intolerant than the people they are criticizing. I think it's better to keep this issue open. |
|
I would like to make very clear in the case of RMS the issue is not merely "intolerance" but also harassment of women and endorsement of child sexual assault. |
Look at it this way. While RMS occupied with open source projects he has less free time to drink and run his dirty mouth on mailing lists. |
|
How about an MVP about just converting the C part to Rust and leaving eLISP as is? |
|
What would be the value of that relative to native emacs? |
|
The core issue is the elisp engine relies on C hacks to get the app loaded. ooof. There is a wild game of pre computing and stashing at the front of the binary. The elisp engine is not clearly separated from emacs. It is emacs. Rewriting the elisp engine is going to be fun because you need to be bug compatible. I deeply loved hacking on this project. I learned a lot. Here is my thought on coming back at this. Start with original C emacs. Try to break it into C "libraries". Learn what code is easy or hard to move. Try to extract just a core C emacs binary as super minimal as you can. This will expose the 40 year old choices that make things hard. Get that solved and it is back to the Rust races. |
What happened?
The text was updated successfully, but these errors were encountered: