Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 9 additions & 10 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,18 @@ name: CI
# Installs the bundle, builds the gem and runs the default test suite on the
# Ruby versions this project validates continuously.
#
# The supported range and the validated set are two different statements. The
# code is written for Ruby 3.3 through 4.0, which is what required_ruby_version
# says; the matrix below runs the ends of that range and the release in the
# middle, rather than every intermediate release. See doc/POLICY.md section 11
# and doc/REQUIREMENTS.md section 20.
# The supported Ruby requirement lives in automatic.gemspec. The matrix
# below is the authoritative set validated continuously; see
# doc/POLICY.md section 11 and doc/REQUIREMENTS.md section 20.
#
# No credential is configured here and nothing reaches an external service: the
# suite must need neither (doc/POLICY.md Invariant 6). Examples tagged :network
# are excluded by default, and none of the Gemfile's optional groups is
# installed, so no plugin's own gem is a condition of this workflow passing.
#
# This is the required check, and it is deliberately the minimal configuration:
# what it proves on every commit is that the framework needs nothing but its
# four runtime dependencies. The all-plugins configuration is a separate,
# what it proves on every commit is that the framework needs only the
# runtime dependencies declared in automatic.gemspec. The all-plugins configuration is a separate,
# non-required workflow, plugins.yml.

on:
Expand All @@ -41,9 +39,10 @@ jobs:
strategy:
fail-fast: false
matrix:
# The continuously validated versions. Keep this in step with the
# supported environment section of README.md; the floor here and
# required_ruby_version in automatic.gemspec are the same version.
# The continuously validated versions. This matrix is authoritative for
# that set; automatic.gemspec is authoritative for the supported Ruby
# requirement. Documentation refers to these sources rather than acting as
# a second matrix.
ruby: ['3.3', '3.4', '4.0']

steps:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/plugins.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
name: Optional plugin dependencies

# Installs the Gemfile's optional `plugins` group and runs the same suite, so
# that the second documented way of setting up a checkout is checked as well as
# described: the group resolves, and the plugin specs it brings in pass.
# that the documented all-optional-plugin checkout setup is checked as well
# as described: the group resolves, and the plugin specs it brings in pass.
#
# This is not the required check. ci.yml is, and it installs none of this: what
# has to hold on every commit is that the framework runs on its own runtime
Expand Down
11 changes: 6 additions & 5 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
# which this file evaluates. Only the optional, plugin-specific gems are listed
# here. See doc/POLICY.md section 9.
#
# `bundle install` with no configuration installs the framework's four runtime
# dependencies and the development ones, and nothing below: a checkout is set
# `bundle install` with no configuration installs the framework's runtime
# dependencies declared in automatic.gemspec and the development ones, and nothing below: a checkout is set
# up to run and to test the framework, not to run every plugin.

source 'https://rubygems.org'
Expand All @@ -16,9 +16,10 @@ gemspec
# and a Recipe that does not use the plugin does not need them.
#
# Every group here is optional, so nothing below is installed by default and
# neither the default test suite nor required CI depends on any of it. Each gem
# is in two groups: `plugins`, which is all of them at once, and one named
# after what it is for, which is one of them on its own. Both are Bundler
# neither the default test suite nor required CI depends on any of it.
# Gems in the aggregate optional-plugin set belong to `plugins` and to a
# purpose-specific group. The group declarations below are the source of
# truth for that membership. Both are Bundler
# groups and both are selected the same way, in the checkout's own .bundle
# directory, which is not committed:
#
Expand Down
92 changes: 48 additions & 44 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ Fluentd, or writes them to a database. Write your own plugin and it composes
with all the others.

**[Follow the Quick Start](doc/QUICKSTART.md)** to install the gem, write a
Recipe that watches four public index pages, install what that Recipe needs, and
Recipe that watches public index pages, install what that Recipe needs, and
produce Markdown from the articles they list.

---
Expand Down Expand Up @@ -98,8 +98,10 @@ exactly three things:
over,
- **a loader** that finds a step by name, so that a Recipe can name it.

Everything else is a plugin. The framework is under seven hundred lines of Ruby
and is meant to stay that size.
Everything else is a plugin. The framework is intentionally kept small; its
responsibilities and architectural boundary are defined in
[`doc/BASIC_DESIGN.md`](doc/BASIC_DESIGN.md) and
[`doc/POLICY.md`](doc/POLICY.md).

It is one person's tooling, run unattended from `cron`, against their own
accounts and their own files. It is not a service, and there is no notion of a
Expand All @@ -115,8 +117,8 @@ a plugin set every part of which still has somewhere to talk to. See

- **Recipes in YAML.** A job is a file, not a program. No Ruby is written to
wire a pipeline together.
- **Plugins across seven categories.** Subscribe, custom feed, filter, store,
provide, notify and publish plugins compose through the same pipeline contract.
- **Plugins organized by role.** Subscribe, custom feed, filter, store, provide,
notify and publish plugins compose through the same pipeline contract.
- **Markdown out of the box.** `PublishMarkdown` writes the result as a plain
Markdown document, to a file or to standard output, with no service and no
credential behind it. It is the natural end of a new Recipe.
Expand All @@ -130,9 +132,9 @@ a plugin set every part of which still has somewhere to talk to. See
- **Retry and interval** on everything that reaches the network, configured per
plugin in the Recipe.
- **A small installation.** A gem needed by one plugin is not a dependency of
the framework: `gem install automatic` brings four pure-Ruby gems and the
command, and installs neither an HTML parser nor a database — let alone an
AWS SDK.
the framework: `gem install automatic` brings the framework's runtime
dependencies declared in `automatic.gemspec` and the command, and installs
neither an HTML parser nor a database — let alone an AWS SDK.
- **No museum.** Every plugin is classified, with its reason, in
[`doc/PLUGINS.md`](doc/PLUGINS.md). Nothing dead is stubbed into looking
alive, and an integration whose service has gone is removed rather than
Expand Down Expand Up @@ -184,7 +186,8 @@ The full account is [`doc/BASIC_DESIGN.md`](doc/BASIC_DESIGN.md).

## 4. Supported environment

- **Ruby 3.3 through 4.0.** CI validates 3.3, 3.4 and 4.0.
- **Ruby 3.3 through 4.0.** The continuously validated versions are the matrix
in [`.github/workflows/ci.yml`](.github/workflows/ci.yml).
- A Unix-like system. GNU/Linux and macOS are what it is used on. Windows is not
supported.
- A compiler only if you install an optional plugin gem that builds from source
Expand All @@ -199,10 +202,11 @@ Two statements, and they are not the same one:
- **Supported range.** The code is written for Ruby 3.3 through 4.0, using APIs
the whole range shares. `required_ruby_version` is `>= 3.3.0` and has no upper
bound, so a Ruby newer than the matrix is permitted rather than refused.
- **Continuously validated versions.** CI runs the ends of the range and the
release in the middle — 3.3, 3.4 and 4.0 — rather than every intermediate
release. A version's absence from the matrix means it is not verified on every
commit; it does not mean it is expected to fail.
- **Continuously validated versions.** The authoritative set is the matrix in
[`.github/workflows/ci.yml`](.github/workflows/ci.yml), which represents the
supported range rather than enumerating every intermediate release. A
version's absence from the matrix means it is not verified on every commit;
it does not mean it is expected to fail.

## 5. Installation

Expand All @@ -213,16 +217,16 @@ gem install automatic
automatic --version
```

That installs the framework, the command and four pure-Ruby dependencies.
That installs the framework, the command and the runtime dependencies
declared in `automatic.gemspec`.
A gem that only one plugin needs is not among them: install it when you use
that plugin, with `gem install nokogiri` or `gem install activerecord sqlite3`.
[`doc/DEPLOYMENT.md`](doc/DEPLOYMENT.md) lists which plugin needs which.

### From a checkout

Use a checkout to try the current development version, change the source,
develop a plugin or verify changes before a release. There are three ways to
set one up; start with the first.
develop a plugin or verify changes before a release. The supported checkout setups are shown below; start with the minimal setup.

```sh
git clone https://github.com/id774/automaticruby.git
Expand Down Expand Up @@ -262,8 +266,9 @@ a Recipe taken step by step through choosing its groups are in
## 6. Quick start

The complete first-run guide is [`doc/QUICKSTART.md`](doc/QUICKSTART.md): it
writes one Recipe that reads four public index pages, installs what that Recipe
needs, and leaves the new articles in a Markdown document.
writes one Recipe that reads the public index pages listed by that guide,
installs what that Recipe needs, and leaves the new articles in a Markdown
document.

```sh
automatic scaffold
Expand All @@ -280,7 +285,9 @@ nothing but the framework and what `gem install automatic` brought.
`feed2console.yml` beside it is the same pipeline printing to the terminal. Read
the file, `grep` it, put it in a repository, or hand it to whatever reads text
next. A Recipe that names a plugin with an optional gem installs that gem first,
which is step 4 of the Quick Start and the habit worth learning early.
which is covered by the "Install what the Recipe needs" section of
[`doc/QUICKSTART.md`](doc/QUICKSTART.md) and is the habit worth learning
early.

To check the framework without any network, write this instead:

Expand Down Expand Up @@ -333,7 +340,7 @@ plugins: # required
- `global.log.level` is the only framework setting. `global.timezone` and
`global.cache` appear in old Recipes and are read by nothing.

Two conventions worth knowing before writing one:
Conventions worth knowing before writing one:

- **Put a store plugin in front of anything with an effect.** `StorePermalink`
records what has been seen and passes on only what has not. It is what makes a
Expand Down Expand Up @@ -392,25 +399,21 @@ like a shipped plugin replaces it.

### Which plugins still work

Every shipped plugin is classified in
[`doc/PLUGINS.md`](doc/PLUGINS.md) section 6, with its settings and the reason
for its status:
The complete shipped-plugin catalogue and the current status of each plugin
are maintained in
[`doc/PLUGINS.md`](doc/PLUGINS.md#6-the-plugins).
The status vocabulary and its meaning are defined in
[Reading the catalogue](doc/PLUGINS.md#5-reading-the-catalogue), and removed
integrations and the reasons for their removal are recorded in
[Plugins that were removed](doc/PLUGINS.md#8-plugins-that-were-removed).

| Status | Meaning |
| --- | --- |
| **Supported** | Works on the supported Rubies with current dependencies |
| **Supported (external)** | Works, but needs something you provide: a service, a command, a credential, a data file |
| **Needs rework** | The service exists; this plugin speaks a replaced interface |

Eleven plugins were removed in this release rather than kept as history: each
talked to a service that has shut down, or through an API that has been
withdrawn with no replacement. They are listed with their reasons in
[`doc/PLUGINS.md`](doc/PLUGINS.md) section 8, and Git history holds the code.
A Recipe naming one of them now fails at load, before anything runs.
This README intentionally does not repeat current plugin totals, per-status
totals, or the current membership of a status. Adding, removing, or
reclassifying a plugin changes the canonical catalogue rather than a
synchronized summary here.

`PublishHatenaBookmark` is currently classified as **Needs rework**; restoring
it to the service's current interface is self-contained work and a good first
contribution.
A plugin classified as **Needs rework** in the catalogue is a
self-contained contribution candidate.

No plugin here is stubbed, mocked or simulated to make a test pass. Where a
plugin's gem is not installed its spec is skipped and says which gem is
Expand Down Expand Up @@ -449,8 +452,8 @@ automatic --version
| `opmlparser <path>` | Print the feed URLs in an OPML file. |
| `log <level> <message>` | Emit one line in the framework's log format. |

The middle five answer "will this work as a Recipe input?" before you write the
Recipe:
The discovery and parsing subcommands help answer "will this work as a Recipe
input?" before you write the Recipe:

```sh
automatic autodiscovery https://example.com/
Expand All @@ -474,8 +477,9 @@ There is no configuration file besides the Recipe. Every setting a job needs is
in the Recipe that defines the job, which is what makes a Recipe portable
between machines.

**Framework settings** — one, `global.log.level`, with the values `info`,
`warn`, `error` and `none`.
**Framework setting:** `global.log.level`, with the values `info`, `warn`,
`error` and `none`. Its Recipe-level contract is specified in
[`doc/PLUGINS.md`](doc/PLUGINS.md#24-global).

**Plugin settings** — the `config` mapping of a plugin entry, passed to that
plugin and read by nothing else. Established names: `retry` for an attempt
Expand Down Expand Up @@ -576,8 +580,8 @@ bundle exec rake
bundle exec bin/automatic -c config/feed2console.yml
```

Contributions are welcome — a new plugin, or reviving the one that needs
rework, most of all.
Contributions are welcome — a new plugin, or restoring a plugin currently
classified as **Needs rework** in the canonical catalogue, most of all.

1. Fork the repository.
2. Write the change, with a spec that reaches no network.
Expand All @@ -590,7 +594,7 @@ judged by: the direction of dependency, where a new capability belongs, how
dependencies are added, how the documents divide, and how a version history
entry is written.

Two rules worth knowing before you start:
Rules worth knowing before you start:

- **A gem needed by one plugin is not a dependency of the framework.** Require
it at the top of the plugin's own file.
Expand Down
5 changes: 3 additions & 2 deletions automatic.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -87,8 +87,9 @@ Gem::Specification.new do |spec|
spec.extra_rdoc_files = ['README.md', 'doc/LICENSE.md']

# Runtime dependencies: what the framework in lib/ requires, and nothing
# else. Requiring `automatic`, loading a Recipe, loading a plugin, running a
# pipeline and the CLI's own work are what these four are for.
# else. Requiring `automatic`, loading a Recipe, loading a plugin,
# running a pipeline and the CLI's own work are what these dependencies are
# for.
#
# A gem needed by a plugin is NOT declared here, however useful that plugin
# is. It is required inside the plugin's own file and installed by the
Expand Down
Loading
Loading