Shape the future of snoots

[thislooksfun]

Thank you for using snoots! It's still under heavy development, so there's a lot of work left to be done. This involves making some hard decisions about how to expose the api to you, and I'd love you all to weigh in with your thoughts!

Think the way I'm doing something is dumb and there's a better option? Is there something missing from an area I've deemed 'complete'? Let me know!

To make tracking these easier, please make separate threads for each piece of feedback.

[OmgImAlexis]

I'm really confused on how this is meant to work.

For example I'm trying to get an array of new posts from a sub but client.subreddits.getNewPosts() isn't a promise.

[OmgImAlexis]

“simple mistakes harder to make”.

You’re over complicating things for the point of making it easier.

The fact that I have to learn a new way of handing the data just to use this library makes the barrier of entry higher than should be. After reading the comments in snoowrap that lead to this I had hoped this was going to be a more straightforward interface but it’s doing just the same, hiding so much that it becomes complicated to work with. Structures like iterables exist for a reason. I’m finding I’m needing to convert the result to one just to use this as I would any other library and I can see this exact issue driving others away.

I promise this isn’t an attack just an observation. I felt like I’ve spent more time trying to workout this then using the library.

[thislooksfun]

Again, if you have a specific use case that the existing solution does not address, please create a discussion so we can address it! I personally have yet to run into a situation where the current solution doesn't make it easier, but if you're fighting it I'm open to changing the design.

My main reasoning is that Listings are not arrays, they are loaded piecemeal so treating them like arrays makes working with them harder imo.

[thislooksfun]

@OmgImAlexis What I'm trying (and apparently failing) to say is that Reddit's Listings are not actually arrays. The Reddit API, in its infinite wisdom, only gives back partial data for virtually every listing request. For example, here is the actual raw response when trying to get the newest posts from r/funny:¹

{
  kind: "Listing",
  data: {
    modhash: null,
    dist: 100,
    children: [
      {
        kind: "t3",
        data: { ... },
      },
      ... 99 more ...
    ],
    after: "t3_mpn4uh",
    before: null,
  },
}

As you can see though it only returns the first 100 items, and gives an after parameter to say "if you want more, fetch it again and say you want the items after t3_mpn4uh". So to get the full list of items in virtually any listing requires multiple API fetches. Because of this there are only 3 options I can see:

1. Fetch the entire Listing every time before handing it to the consumer. This is a bad idea because often people either don't care about the listing at all (think comments on Posts or the replies of Comments, if you only care about the body why waste effort fetching data that won't be used?) or only care about part of it (like looping until it finds something it's seen before).
2. Make the user fetch more when they need it. This is the pattern snoowrap uses, and I find it very clunky and error-prone, as described in a previous comment.
3. Make an opaque interface that cleanly iterates through the items without the user needing to worry about partial listings.

Making Listings into an iterable would necessitate one of the first two options, both of which are flawed and have more downsides than upsides. Option 3 (which is what snoots does) does make using array-based interactions impossible without an intermediate step, but I'm struggling to think of any real use cases that can't be covered by the existing api. What I meant by "if you have a specific use case" was "please share your code so we can see if there's anything I can do to help you clean it up", be that through a refactor on your end or a new api method on Listing.

My goal here is to make a more straightforward interface, and I strongly feel that hiding the messy (and quite frankly ugly) interactions needed to make Listings work is part of that.

---

¹ For the full response, you can see here, but be warned: it's very long.

[thislooksfun]

I wrote up a (hopefully) more detailed post over here. I'm going to lock this discussion as I never intended it harbor such in-depth discussions. If you want to continue this further (and I would encourage you to do so, since it seems you have legitimate feedback!) please open a new discussion. That allows me to keep track of the conversation easier, and also mark when it has been resolved, which I can't do to a reply chain.

[thislooksfun]

@OmgImAlexis I'll admit it, I was wrong. I thought that iterators had to be synchronous, so it would be impossible to make work with the limitation at hand. I was completely unaware of for await and Symbol.asyncIterator. Without the existence of for await I think that my conclusions were valid, but since it exists I agree that it should be available. I'm in the process of implementing it and updating the docs now.

EDIT: Listings are async iterable as of v1.0.0-dev.9

[eli-rich]

When fetch becomes non-experimental, is there a plan to use that instead of got? Cuts out a dependency.

[thislooksfun]

There is now! I didn't actually know Node was adding a first-class fetch() implementation, so thanks for letting me know!