C bindings?

[rempas]

Hello! I just found out Louvre from this post and I find it very interesting! The benchmarks also look very interesting!

I would like to ask if you plan to create C bindings?

[ehopperdietzel]

Hi, thank you. Creating C bindings might be challenging due to the use of virtual methods. However, one approach could be to create a function pointer for each virtual method and then have the virtual method call that function. I'm not sure if it would be the best approach, but I'll take some time to think it over. If you have any ideas, I'm all ears.

[rempas]

What? Are you creating your own language? How cool, hahaha. I saw the examples in your repo,

I'm glad you find it interesting. I suppose, you mean the files in the code_examples directory? These are just some snippets for things that I don't want to forget and also don't have a place in documentation for yet. The real beast is under docs/book! It is made with Sphinx. It also contains the "ROADMAP" and "LANGUAGE SPEC" files written in Asciidoc. It's not by any means the "final" design of the language and everything may change until the language hits stable 1.0 version, but you can check it and have some fun! ;)

and it has an interesting syntax, like QML + C++ + TypeScript?

The Syntax is mostly the "classic" C based syntax but with extra things that I hope to fit together in a good ecosystem. In general, I don't change things just to change them and say that I did something different, and I don't keep things the same because they have been that way for years, and they are "the standard" and what people know. I want to create something that will people will like and see need of using. For that reason, I must be very open-minded with its design and not do things just to do them.

And how are you creating it? With Bison, Flex, and all that?

Here's the crazy part... I'm not using anything! Not even Libc, except for "opendir", "stat" and "lstat" and that's because for some reason, I cannot make these work by myself! I know that most people will say that I'm "crazy" or even that I waste my time but for me, it was very important to feel confident and satisfied with the tools I would use as if you read the documentation, you'll understand what kind of beast I want to create (and it's still not full yet) so my job should be as easy and enjoyable as possible.

I have spent a lot of time writing and testing the system library, but I'm now done, and I'm ready to spend even more time writing the parser, lol! Also, another positive thing from building the library is that, I have it ahead of time and this is the same library (API) that will be the standard library of Nemesis. So I (we) can find any bad parts about it faster and finalize it sooner, so when Nemesis is out, the standard library will be there, and it will be battle tested! At least, parts of it...

Overall, I would say that I'm very happy that I choose the route of fully writing everything by hand. Until recently, I was even thinking of creating my own backend (not in assembly but directly in machine languge), but I wanted to by realistic with my goals and time as writing a backend is another project by itself! For that reason, I will start with LLVM and MIR and maybe add my own backend in the future (especially if someone is interested to help and teach me some things). But if MIR adds support for shebangs and system calls (and maybe some others things that I don't know now), it may not be necessary unless someone wants to write a kernel in Nemesis, and we need a backend that doesn't depend on libc.

Is it pre-compiled?

I'm sorry, I don't understand that. Are you asking if it's a compiled language or an interpreted one?

Regarding virtual methods, I used them for two reasons. First, for providing default implementations (yes, I know I could have simply used function pointers). But there's a second reason, simply that being forced to create your own class for Surfaces, Toplevels, etc., results in a well-structured project. So, anyone who sees your code and knows about Louvre can immediately understand where everything is being handled. To port it to C, I imagine I would be forced to use function pointers, but I think there might be a better solution; I just need to do some research ;)

That's interesting! I do read that, and I think that I understand it, and I cannot see why you cannot do your job just fine (if not better) with Polymorphism. Maybe either I don't understand exactly what you are trying to do, or I don't know C++ enough, and it will not work as I expect it. So, I have tried to create the following example to do what you described using Polymorphism.

#include <stdio.h>

// Lourve library file
class Parent {
public:
  const char* name;
  int         age;
};

class Surface : public Parent {
public:
  const char* surface_name;
};

class Toplevels : public Parent {
public:
  int amount;
};

// This function will be used for "Parent" and any object that inherits from it
void show_parent(Parent& p, const char* type) {
  printf("%s info:\n", type);
  printf("  name: %s\n", p.name);
  printf("  age: %d\n", p.age);
}

// This function will be used specifically for the "Toplevels" objects (and any
// class/struct that inherits from it) instead of using above function function
// This effectively works like overriding a method
void show_parent(Toplevels& t, const char* type) {
  // Cast the type so you don't duplicate code ;)
  show_parent((Parent&)t, type);
  // Add the rest of the code
  printf("  amount: %d\n", t.amount);
}

// These is the function calls that non C++ languges will use (no function overloading is allowed!)
extern "C" {
  void show_parent_parent(Parent* p,       const char* type) { show_parent((Parent&)*p, type); }
  void show_parent_toplevels(Toplevels* t, const char* type) { show_parent((Toplevels&)*t, type); }
}

// Devloper (library user) file
class ThirdClass : public Toplevels { // Sorry, I don't have it with names, XD
public:
  int amount2;
};

int main() {
  Parent p =     { "Parent",     40 };
  Surface s =    { "Surface",    10, "Cool Surface" };
  Toplevels t =  { "Toplevels",  15, 3 };
  ThirdClass c = { "ThirdClass", 20, 5, 7 };

  /* C++ project will call these */
  show_parent(p, "Parent");     printf("\n");
  show_parent(s, "Surface");    printf("\n");
  show_parent(t, "Toplevels");  printf("\n");
  show_parent(c, "ThirdClass"); printf("\n");

  /* Non C++ project will call these */
  show_parent_parent(&p, "Parent");        printf("\n");
  show_parent_parent(&s, "Surface");       printf("\n");
  show_parent_toplevels(&t, "Toplevels");  printf("\n");
  show_parent_toplevels(&c, "ThirdClass"); printf("\n");

  /* Of course, it is worth mentioning that a Non C++
   * project could also use the mangled C++ symbols
   * if it knows their names ;) */
}

Will this do what you want to do? I have compiled and run it, and it seems to work as expected! Of course, one thing that I forgot before is that C++ does not support UFCS so you'll have to call them as functions. That may be one thing to think about...

Let me know if I don't understand something because I truly care about Louvre and even tho I cannot help with code, I want to help with anything else that I can. So, I don't just provide ideas to do just say them, but because I truly think that they can improve a project!

P.S. Sorry if I messed anything in that long reply. It's almost midnight at that point and I just wanted to finish, so I can close everything and go to sleep...

[ehopperdietzel]

The real beast is under docs/book!

I took a look, and there are several brilliant ideas that I haven't seen in another language, like doing var.method() if the variable is of the same type as the argument of some function. Hahaha, I found it ingenious. The loops also look interesting. Can it be used already? And would it be compatible with C code?

Here's the crazy part... I'm not using anything!

What? Sounds like quite a task. But does that mean you have to create machine code for each architecture? Or are you targeting a specific one? Honestly, I don't know much about language/compiler design and all that. I imagine something simpler might be preprocessing your code to C and then using a C compiler. Is that what you're doing, perhaps?

Not even Libc, except for "opendir", "stat" and "lstat" and that's because for some reason, I cannot make these work by myself!

Curious, I had never wondered how those functions work internally. I suppose to reimplement them, you would have to make very low-level calls to Linux APIs? Honestly, I don't know much about it.

Are you asking if it's a compiled language or an interpreted one?

Exactly

Example Code

Your example code is the same as I imagine for setters/getters. The issue arises with virtual methods.

What Louvre does now is:

class LouvreClass
{
public:
    virtual void someEvent()
    {
        // Library does something here by default
        printf("LouvreClass");
    }
}

class MyLouvreClass : public LouvreClass
{
public:
    virtual void someEvent() override
    {
        // Override
        printf("MyLouvreClass");
    }
}

...

MyLouvreClass *myInstance = new MyLouvreClass();
myInstance->someEvent(); // Out: MyLouvreClass

// Cast to LouvreClass
LouvreClass *instance = (LouvreClass*)myInstance;
instance->someEvent(); // Out: MyLouvreClass

Note that even if the instance is treated as a LouvreClass, it still prints "MyLouvreClass". If someEvent() weren't virtual, it would print "LouvreClass" instead. The library internally handles the base classes to invoke their events, so they must be declared as virtual.

The solution I'm considering is as follows:

static void someEventDefault(LouvreClass_c *ref)
{
    printf("LouvreClass");
}

class LouvreClass_c : public LouvreClass
{
public:
    void (*someEvent_c)(LouvreClass_c *ref) = &someEventDefault;

    virtual void someEvent() override
    {
        someEvent_c(this);
    }
}

And then create functions, as you propose, to allow the developer to replace the callback function. But of course, it would introduce additional overhead.

[rempas]

I took a look, and there are several brilliant ideas that I haven't seen in another language, like doing var.method() if the variable is of the same type as the argument of some function. Hahaha, I found it ingenious. The loops also look interesting.

Well, funny enough, this is possible in some languages like D or Nim! It's called Uniform Function Call Syntax, and it is indeed very interesting! It will be especially useful in Nemesis where there will be no "methods" but just function that take objects as their first arguments. I am re-thinking about not implementing support for using something else other than the first argument before the dot, however, as it doesn't seem very practical, and it can lead up to confusion...

If you want to see about some very interesting concepts and ideas and don't feel like reading the whole docs, take a look at the following:

• Iterators
• Scopes
• Nested assignments
• Conditional assignments
• Conditional branches
• The token operator ($)
• Type arguments
• Pointer mutability system
• Pointer arithmetic
• The whole object chapter, lmao!

There are a lot of interesting things that I haven't documented yet, like the Error system, the test suits and the built-in forks/threads, the bitfields and a couple of others!

I also plan to provide a combination of both compile-time and run-time (with C++ like smart pointer tactics) memory management that will also allow you to choose your own memory allocator if you don't want to use the one from the standard library. And of course, I haven't written a documentation about the package manager, that will be another beast by itself! Both of those are yet to be designed, however!

Can it be used already?

At that point, I have finished the library (only threads remain, but I will not implement them until it's time to support modules) and the CMD arguments. When I find time and by in the mood, I will start working in the parse and the Intermediate representation. I plan to release the language little by little, creating the roadmap with the wanted features. Things can of course change at any moment, as I am also very open to suggestions and even contributions.

And would it be compatible with C code?

I suppose, you mean if we would be able to directly import and compile C files (including headers)? If yes, then the answer is no. Of course, that would mean that I would need a C frontend and that's another project on itself (even tho C is considered as a "simple" language). So, I again need to be realistic with my powers and time and set goals.

The good news however is that this is something that I have think about in the past and a couple of other languages support that (D and Zig coming to mind) so at some point, I will start working on allowing that! But in the beginning, C projects will need a C compilers and bindings need to be written manually!

What? Sounds like quite a task.

Yep, it is! But tbh, tell me a program that isn't a big task, haha! Seeing people been excited about it gives me motivation to work on it! ❤️

But does that mean you have to create machine code for each architecture? Or are you targeting a specific one?

Well, like I said, that was the plan in the beginning! However, I wanted to be realistic and blah blah blah, you already know! Creating a backend will require the following tasks:

• Learn machine language. There aren't a lot of resources, and it's a very hard task by itself. Keep in mind that you would also need to learn a new "language" for every CPU ISA you wanted to support. So yeah, that will take some months, if not years...
• Learn how to write good and optimized code. Learning machine code is one thing. Learning techniques about how to optimize the high level code is another hard task! This is what separates compilers like Clang and GCC with compilers that do very little (or not at all) optimizations like TCC (of course, TCC is an awesome compiler for what it does!).
• Learn how to create the files that need to be created. Different OS may have different files (depending on the OS). Unix have ELF, Windows has PE and MacOS has Mach-O. So, another couple of months (if not years) to support them all properly.

In sort, creating a backend is another titanic project by itself! For that reason, I have decided to use LLVM and MIR. These two are backends, meant to be used by compilers to create the "final product". They are libraries that have their API in the form of an Intermediate representation. That way, I don't have to do all the task mentioned above. There are some negative things on having to use a backend but at least, for now, they are worth it!

Honestly, I don't know much about language/compiler design and all that.

Hey, Pssst, Come close! I'll tell you a secret.... I don't either! I mean, other than understand the whole concept and how things work in theory, I haven't read a book or another compiler! I did try to read a book, but I got bored and find it very repetitive. I didn't want to create Nemesis just because I'm not 100% happy with any language that there is out there (and I think that we can do much, much better!) but also because writing a language is a project I want to do! I don't care if it gets popular or "big", the most important thing is to become a better programmer and have lots of fun doing it! For that reason, Nemesis is build with love above all! That's why I welcome everyone who wants to embrace it and be part of a loving community!

I imagine something simpler might be preprocessing your code to C and then using a C compiler. Is that what you're doing, perhaps?

This technique is called transpiling. This can be useful in a number of situations, but I do not find any usage for it in the current case. The only thing it provides is the fact that I'll have a high level language as an intermediate representation, but even that is going to be a double edge sword because it will also increase the complexity as I will have to translate things to the way C does things. This technique will also increase the compilation times by A LOT! Hence, why I didn't choose it.

Curious, I had never wondered how those functions work internally. I suppose to reimplement them, you would have to make very low-level calls to Linux APIs? Honestly, I don't know much about it.

Yep! Eventually, I'm going to have to make system calls. You can think of system calls as "kernel functions". Essentially, they are calls (that are made in the machine instruction level) that request a functionality from the kernel. Imagine things like:

• Reading from and writing to a file
• Asking information about the current process
• Asking information about the system (eg. CPU cores, Number of devices that are connected)
• Creating a new thread or fork the current process
• Run a new process
• And tons more!

Linux is my priority, but anyone is welcome to contribute and give support for other OSes as the project grows. On the positive note, this will be very easy for Unix based Operating Systems as they have similar APIs for most of the stuff. It will however be harder for Windows as it doesn't closely follow the Unix API. But Windows will also work out of the box with WSL. FreeBSD also has a Linux compatibility layer, so it will also probably work out of the box!

Are you asking if it's a compiled language or an interpreted one?

Exactly

As you probably understood, it will be a compiled language. I also want to give support for interpretation and JIT in the future. However, it will not be the "classic" architecture of an interpreter.

Your example code is the same as I imagine for setters/getters. The issue arises with virtual methods.

What Louvre does now is:

class LouvreClass [ ... ]

Note that even if the instance is treated as a LouvreClass, it still prints "MyLouvreClass". If someEvent() weren't virtual, it would print "LouvreClass" instead. The library internally handles the base classes to invoke their events, so they must be declared as virtual.

The solution I'm considering is as follows:

static void someEventDefault(LouvreClass_c *ref)
[ ... ]

And then create functions, as you propose, to allow the developer to replace the callback function. But of course, it would introduce additional overhead.

All right! I got that now and I see! With overhead, you mean that fact that you will have some extra "extern C" functions that will call your C++ functions?

[ehopperdietzel]

It's called Uniform Function Call Syntax

Well I learned something new

I am re-thinking about not implementing support for using something else other than the first argument before the dot, however, as it doesn't seem very practical, and it can lead up to confusion...

Hmm, I'm not sure if I understood. For example, would this be supported?

bool contains(const char *str, const char *substr);

...

const char *text = "Is this supported?";

if (text.contains("supported"))
{
}

Scopes

I've been looking at the documentation, and I like all the ideas. There's just one detail that caught my attention, which is treating a 'break' as if it were an error. Many times, I use 'break,' for example, when I'm searching for a value in an array, and therefore, I don't associate with an error. Perhaps it could be replaced with another word? Just a suggestion

I plan to release the language little by little.

Well if you need someone to try it out, just ask me ;)

I suppose, you mean if we would be able to directly import and compile C files (including headers)? If yes, then the answer is no.

Oh :( that would have been great.

For that reason, I have decided to use LLVM and MIR. These two are backends, meant to be used by compilers to create the "final product". They are libraries that have their API in the form of an Intermediate representation

Again, I am learning a lot. Thank you! Hahaha

You can think of system calls as "kernel functions"

I've played around with creating some Linux drivers (which provide userspace APIs), so I think we're on the same page.

Linux is my priority, but anyone is welcome to contribute and give support for other OSes as the project grows

Have you considered sharing this project on forums like Reddit, Lemmy, etc.? I'm sure you would find people who could help you with that and contribute ideas.

[rempas]

Hmm, I'm not sure if I understood. For example, would this be supported?

bool contains(const char *str, const char *substr);

...

const char *text = "Is this supported?";

if (text.contains("supported"))
{
}

Yeah! This is the basic functionality. What I was talking about is something like the following:

use erebos.file

void greet(string name, u8 age) {
  /* "write" writes to a file (stdout if not given) with a formatted way (inspired by Rust) */
  write("Name: {}\nAge: {}", name, age)
}

void start() {
  string name = "John"
  u8 age = 28

  greet(name, age) // Normal function syntax
  name.greet(age) // "name" is of type "string" so the first argument is used. The rest follow inside paranthesis.
  age.greet(name) // "age" is of type "u8" so the second argument is used. The rest follow inside paranthesis.
}

This is how it will work! This is already present in the docs, but I have added a note (in the next commit) that specifies that it isn't planned to be used as it seems confusing and pointless so only the first argument will be able to be used just like all the other languages that support UFCS!

Could you think of a case it could be useful? If you or anyone else can provide useful examples until I implement the functions (you have a lot of time, lol), I will support it!

I've been looking at the documentation, and I like all the ideas. There's just one detail that caught my attention, which is treating a 'break' as if it were an error. Many times, I use 'break,' for example, when I'm searching for a value in an array, and therefore, I don't associate with an error. Perhaps it could be replaced with another word? Just a suggestion

Done! scope(failure) is now scope(break)! That was a great idea, please make sure to give more in the future! If and when you want, of course ;)

Well if you need someone to try it out, just ask me ;)

Oh, you must be really interested! Yeah sure, I just don't know how to contact you. I checked your profile on GitHub and there isn't an email or something. Am I missing something? Also, you could star and follow so you can get updates and try them out. I do try to give commits that say what I change/implement. As much as I love Nemsis, I feel like I "relax" and "cope" with it and I plan to learn Svelte to build an app that I want and hopefully make some money! Nemesis will stay in the corner for some time, unfortunately...

Let's think positively and be excited about the future however! In the next commit, I will also create a roadmap and a changelog, so things will be more clear. I'm also thinking about doing the "less is more" approach and push commits only when I implement features and have each commit be a different version (of major 0). Tho, I'll rethink about that, as it makes things less fun and creates bigger time periods between commits. What do you say?

Oh :( that would have been great.

It would! I want it as well, as this will make using C (and C++ ones that use "extern C") projects or bindings much easier! Perhaps, until the other priorities are done, and it's time to work on it, I could look if there are other C parsers than can be used that way I need them. ;)

I don't promise anything, but we'll see! For sure, however, it is planned to be implemented at some point!

Again, I am learning a lot. Thank you! Hahaha

My pleasure! I have learned a lot from others and I have still tons of things to learn! Sharing knowledge and helping each other is the best! People who don't do it miss a lot, lol!

I've played around with creating some Linux drivers (which provide userspace APIs), so I think we're on the same page.

Oh, that's so cool!!! I was thinking to mess with the Linux kernel to provide some systems calls, but I can't even get to make a custom kernel boot in my Arch (btw) machine after all these hours trying different things. 🤦

Have you considered sharing this project on forums like Reddit, Lemmy, etc.? I'm sure you would find people who could help you with that and contribute ideas.

Oh, Lemmy seems cool, thanks for the suggestion! I am interested to make accounts and work only on networks that are backed by open source tools (and maybe some small, transparent and safe closed source ones)! This is why I also host on Codeberg. I know this will limit the popularity and have it been discovered slower, but like I said, I mostly do it to have fun and create something beautiful... And because I can! 😎