User talk:DKinzler (WMF)/API URL Guidelines

Conversation with Moriel

[copy of a conversation on Slack, for reference]

Moriel: btw, I was thinking a lot about your point with the component/domains and I agree on the separation of them but I think the problem we have (generally at the movement btw) is that I'm not sure we know WHAT the domains ARE

like... i'm not sure there's agreement between people, or that it's super understood. I thikn Dana has great exercises to flesh domains out that we can use -- because domains can also include sub-domains, and some could have one component but some could have multiple

The Amazon example really exemplifies that; they are clearly going by domains there -- but not necessarily strictly "team components" because Ads API for example 100% has multiple teams that work on different components, but it's a domain... we don't have to follow that in terms of "only domains have upper URLs and versioning" but it's good for us to kinda map and know the difference (edited)

also holy crap regarding their lack of consistency

Google's APIs seem to be more or less where you're going with things, I think? Domain separation, allowing for sub-domain/component separate versioning ... except let's make sure we do it at least somewhat consistent (/v1 for all or :v1 for all, whatever, just... let's pick one)

Daniel: In practice and in the short term, I don't really care about the domain boundaries being "right" at the moment... for now, in most cases (all except MW core), the component would correspond to a code repo. Versioning is bound to deployment, and deployment is bound to repo structure.Of course, repo structure should follow logical domain boundaries. But figuring out the "right" boundaries in the abstract, and using them in the URL structure, would do more harm than good if they do not match repo structure and team structure.

Moriel: Hm. But if we're already doing this, shouldn't we do this strategically? I mean... I don't mean we take 5 years to figure it out, but it would be nice if we can get something that follows our understanding of how things SHOULD be divided even if we start out with an MVP that doesn't entirely do it, at least so we have a vision for this and can iterate.I worry otherwise this will become one more thing we need to fix in a year or two, OR that since the team managing this is a bit ambiguous (is it the MWAPI team?) then it will lose control and be messy... the more clarity, the better.

Daniel: I agree that Amazon's Ad API has multiple teams working on it - but it probably has multiple subdomains. URL structure should reflect team structure in a way that it should be easy to find a base URL for which you can say that a single team owns everything under it.

I agree that we should start with an aspirational domain model for mw core, so we don't lump everything together. Most other things have mostly-sensible boundaries already.Once we have good versioning practices and deprecation mechanism in place, we can iterate on the structure pretty easily.

The problem we have now is that multiple teams have put various kinds of stuff under the /page/ prefix, because it all relates to pages somehow. I fear that if we go with "conceptual" domains, we will see more of that problem.

It seems beneficial to me if there was one team responsible for, say, /api/accounts.v2. If login is managed by a different team,let it live under /api/login.v1. If it's the same team, it can go into /api/accounts.v2 as well.

Moriel: I am not convinced URL should reflect team structure at all. Our team structures change and to outsiders no one really knows (or cares?) what our teams ARE.I agree that we need to find the balance in giving people freedom to iterate without getting stuck on versioning from others, or as little of this as possible, but we can find the middle ground by making those based on the subdomains that make sense, regardless of what team works on that this week. Even if you have a versioning-schema that includes a couple of components you can still be logical (and not-stuck) in how you iterate on them AND you maintain the structural integrity of the API regardless of the teams' weeks flavor AND regardless of who is maintaining a component this week (including volunteers)

I don't know how easy it would be the iterate on structure even with deprecation and versioning process. It won't be impossible, but it won't be straight forward either. We won't be able (or .. it will be very hard) to suddenly go "actually we have completely different entrypoint/component distribution for the URL because we're changing our minds re domains)I think a LITTLE bit of this will happen regardless, but the difference would be where w're coming from -- a lot of this will involve "teaching" or getting buy-in to the "why is this this way" and if we have a solid grounding it will make more senseBut I also think we can meet somewhere in the middle on this

Daniel:

> subdomains that make sense, regardless of what team works on that this week.

Absolutely, as long as it's only one team at a time.What I want to avoid is that more than one team works on endpoints under the same prefix. They would have to function as a single team with respect to the component - possible, but painful.

> I am not convinced URL should reflect team structure at all.

I only want it to reflect team boundaries in the above sense: if different teams own it, it should not be versioned together, and should have separate specs.But I an see how we don't want to have too many components, all versioned separately. I think that a single team needs to be responsible for the spec, release cycle, and cohesion of the component, even of multiple teams work on the implementations. And, one level up, we'd want one team to be responsible for more high level consistency across all components. (edited)

> a lot of this will involve "teaching" or getting buy-in to the "why is this this way" and if we have a solid grounding it will make more sense

Exactly, and I want to start the conversation ASAP

I think API migration doesn't have to be difficult or painful. But getting to the point where it isn't will take some work and patience. Part of it will be writing client libraries that make it easier for API clients to do the Right Thing.

> But I also think we can meet somewhere in the middle on this

I am sure we will! DKinzler (WMF) (talk) 10:25, 27 July 2023 (UTC)

Conversation with Giuseppe

[notes from a conversation withGiuseppe]

• Agreed: we should not expose implementation details like rest.php. rest_v1 for restbase endpoints was an ad-hoc solution to avoid /api/v1 getting "hogged" by restbase.
• Agreed: "routing prefixes" are a good way to distinguish APIs with different operational needs and guarantees, as well as public/stable from internal/restricted APIs.
• Guiseppe: we should have a policy that gioverns API (URL) design, but we also need to have clarity on who can make exceptions to that policy. Do we just want to stop the bleeding, or do we want to deprecate old URLs to consolidate existing APIs? Needs product vision.
• Giuseppe: fine grained per-component versioning makes sense for internal APIs, not practical for public APIs. All public APIs under a prefix should be versioned together. So e.g. /wmf-api/content.v7/page for an internal API, but /api/v2/page for a public API. Need proper product management of public API.
• Giuseppe: API versions are different from library versions, because old versions of the API vanish (after a year or so), while old versions of libraries stay usable for a decade or more.
• Giuseppe: the idea of api.wikimedia.org as the entry point for all APIs was a mistake, most APIs should be per-wiki.
• Daniel: How does that translate to third party installations and extension APIs? The distinction between "extension" and "service" is going to get increasingly blurry.

DKinzler (WMF) (talk) 10:38, 27 July 2023 (UTC)

Conversation with Tim

[notes from a conversation Tim]

• Agreed: "routing prefixes" are a good way to distinguish APIs with different operational needs and guarantees, as well as public/stable from internal/restricted APIs.
• Agreed: prefer per-component versioning also for public APIs. Without them, teams have to wait for the next cross-API version to roll around, which probably doesn't happen very often, because it's hard. Current experience shows that it doesn't happen at all.
• Agreed: don't use "core" as a component.
• Agreed: Components should be owned by a single team. Can align with extension names and service names. MW core can be split into a handful of sensible domains.

DKinzler (WMF) (talk) 12:08, 27 July 2023 (UTC)

Conversation with Timo

[notes from a conversation I had with Timo]

• Need to make sure that whatever pattern we establich will also work for 3rd party wikis. Must not be WMF only.
• One notabler type of "internal" APIs are APIs used by MediaWiki itself. Since the provider and (intended) conumer of these APIs are versioned together (my MediaWiki release), they can evolve more quickly than "public" APIs. In contrast to WMF specific "internal" APIs, these would exist on third party wikis as well.
• On third party wikis, we really only have the /w/rest.php prefix available at the moment.
• For the future, we will need a mechanism to plug in stadalone API providers as part of (or instead of) "extensions".

DKinzler (WMF) (talk) 20:11, 14 August 2023 (UTC)