Review bot?

[egeerardyn]

After seeing #682 (comment) (and quite a bit of similar PRs ), I think the only solution we have left to reduce people from inconsiderately submitting a PR, is by automating a part of the review process.

In principle, it should not be hard to write a small bot that runs e.g. on Travis CI to check:

• that all commit titles in a PR are not just "Update readme.md",
• that the repository is added at the bottom of a category,
• that a new repository:
  • is e.g. older than 2 weeks,
  • contains the awesome badge,
  • contains a "CONTRIBUTING.md", and
  • contains a "LICENSE.md" or similar file.

Those are all mechanic checks that such a bot could carry out instead of a human being. Probably we can make this bot even react to the GitHub PR with an error message and even close the PR: personally, I would be quite ruthless when it comes to handling such inconsiderate PRs: if it is too hard to read some instructions, I am pretty sure it is too hard to maintain an awesome list.

The danger of this idea are false positives: people that are of good faith who get their PR closed because of this.

[JesseTG]

Then they can reopen their PRs.

[sindresorhus]

Would be nice, though I'm sure people will find many other ingenious ways to do it wrong that the bot won't catch.

[egeerardyn]

@sindresorhus I am sure that people will find both

• ways around such a bot (I can think of a few myself), and
• ways to mangle their list such that a bot cannot decide whether a list is good or bad.

However, I don't think that would be a problem. I see such a bot as a spam filter. Even if it doesn't get rid of 100% of the problem, it still is an improvement: it could take over a few simple repetitive tasks such that we free some time and energy for the human review process.

[JesseTG]

ways to mangle their list such that a bot cannot decide whether a list is good or bad.

Then the list is probably too mangled to be useful for awesome anyway. :)

ways around such a bot (I can think of a few myself)

Cool, let's hear them.

[sindresorhus]

Don't get me wrong though. I'm all for a good bot.

[sobolevn]

Any progress yet? I am very interested also. And can help for sure.

[egeerardyn]

@sobolevn Haven't started yet: I currently have bigger priorities for work.

[jacc]

I feel extremely bad for necroposting on a issue over a year old, but opening another issue didn't seem practical. I could look into creating a CLI app that the users run before contributing to check if it has the correct files / badges, etc.

[ntwb]

Sounds like an app for https://probot.github.io/ would be ideal, could be used by all awesome repos

[jacc]

@ntwb I could definately work on something like that. Probot's API seems super simple. All I have to do is just get the repo information to see if it has the Awesome badge in it.

[jacc]

@ntwb @sindresorhus I've started development of the bot at https://github.com/jacc/awesome-bot. Haven't committed in a while but I will soon with the new features.

[ntwb]

Great @jacc 👍

---

As an FYI, I came across this issue in my search for an issue that was looking to standarize pull requests, my thought was and is that if PRs were standardized across awesome repos then it would make my life easier as a watcher of ~12 awesome repos.

Primarily I'd love to see a consistent format of URLs in PRs so when I see a new notification in my email the link is easily clickable so that I can checkout the latest link being added to a repo without having to juggle copy and paste, one click, done 😬

[fkromer]

There is also Danger to validate PRs with awesome_bot.

[sindresorhus]

Use https://github.com/sindresorhus/awesome-lint in your Awesome lists.

We plan to add it here too: #1394