CPAN Meta Spec Proposals

From Perl QA
Jump to: navigation, search

Contents

Introduction

NOTE: The final release of Version 2 is on CPAN. See CPAN::Meta::Spec. Thank you to all who contributed ideas and comments. This page is preserved for the historical record and should not be amended with new proposals.

History

Note: the CPAN Meta Spec is now maintained in a git repository. See the cpan-meta-spec repository on github.com for details.

Dagolden is organizing a working group to review, triage and implement a new spec. The goal of this effort is incremental improvement in the CPAN Meta Spec, not a complete re-write. Priority will likely go to suggestions that:

  • Improve clarity or resolve ambiguity
  • Are narrow in scope
  • Address specific, current deficiencies
  • Require minimal (volunteer) effort to implement in the toolchain.

Formatting and schema

Update the YAML version declaration

Proposal

The version declaration for YAML has not been "--- #YAML:1.0" for a long time, update it to the current YAML 1.1 equivalent.

Comments

Formally switch to "YAML Tiny"

Proposal

Change the language referring to "YAML" to instead refer to "YAML Tiny" and update the specification URL to point to the YAML Tiny specification. (AdamKennedy)

Comments

"YAML Tiny" is a subset-specification in the style of the "SVG Tiny" specification [1]. It just happens to be in a module called YAML::Tiny, but a YAML Tiny parser is in the core as Parse::CPAN::Meta. This parser also adds a few illegal edge cases like the current illegal #YAML:1.0 version declarator (which no longer looks like this).

The creation of configure_requires stalled because of the need for a YAML parser in the core to parse META.yml. All current full YAML parsers were buggy or too large, and we don't use most of the features of YAML anyway. The result was that the 5.10.1 contains a parser tailored to the needs of META.yml. (AdamKennedy)

Deprecate YAML Tiny dialect for JSON

Proposal

The current META.yml isn't really YAML. It has many of the drawbacks of YAML and none of the technical benefits. It has the benefit of being sort of user editable, but both YAML and YAML Tiny can seem more human-editable than they are. We almost certainly want machines writing META files anyway.

META.json should replace META.yml over time. Given both, META.json should be preferred. JSON is entirely unambiguous, very easy for humans and machines to read, has a very short and clear specification, and has a very well proven pure-Perl reader and emitter with no non-core prereqs that could be cored. Parse::CPAN::Meta could handle both.

Alternately, META.yml could be detected as either JSON or YAML Tiny by Parse::CPAN::Meta, relying on the "well, JSON is YAML" hand wave.

Comments

  • I would like to phase out the use of YAML Tiny entirely. YAML was a great idea that has failed to thrive in Perl land, and YAML Tiny is a great way to cope with our reliance on that failed idea. Rather than let YAML Tiny live forever, we should begin the process of deprecating it entirely over the next 5-10 years. rjbs 15:36, 28 August 2009 (BST)
  • I concur with Ricardo that at the very least we should at least state in the next version that the YAML format is ultimately destined for deprecation. AdamK
  • I would like one violation of the JSON spec: allow Javascript-style comments. My one beef with the JSON spec. Elliot

Formalize allowed version number formats

Proposal: Formalize the spec for version numbers as "decimal or normalized dotted-integer (leading "v" and at least 3 components). (Dagolden)

Comments:

  • It's not clear whether/how to include alpha versions (with underscores) in the spec as they generally cannot be resolved by the toolchain (Dagolden)
  • Version comparison for "greater than" must be both possible and fully described in all possible combinations, as things like Module::Install need to handle duplicate dependency assertions. For example, part of the M:I internals silently assert "requires File::ShareDir 1.23". If the user then manually asserts "requires File::ShareDir v1.24.1" M:I needs to be able to determine which to keep and which to discard. (AdamKennedy)
  • SUPER 1.16 declares some dependencies as the empty string instead of zero. This apparently causes problems for some part of the toolchain. We should clarify that empty string (and undef) are not allowed. I believe the toolchain special cases 0 to mean "any" even if a module does not define $VERSION, but we should specify that behavior explicitly as well. (Dagolden)

Schema

Proposal:

The META spec should come along with a formal schema definition. (SlavenRezic)

Comments:

There are at least three schema languages available for this task, all with a Perl implementation: Kwalify, Data::Rx, and Data::Schema.

For the current META 1.4 spec there is a Data::Schema schema, packaged as a Perl module Data::Schema::Schema::CPANMeta but actually the schema itself is written in YAML [2]. User:Stevenharyanto Nov 18 16:32:44 WIT 2009

For the old META 1.3 spec an attempt to define a Kwalify schema [3] exists. As you can see from the embedded comments, some parts of the current META spec are not clear enough.

Note that Kwalify does not define a "any of" type (but Rx and Data::Schema do). This type would be necessary if definitions like "author may be a scalar or a list" would be allowed. I am willing to implement any-of as a non-standard extension to Kwalify. But maybe the META definition should be as simple as possible?

Data structures, not YAML

Proposal:

The META spec should be defined in terms of (Perl) data structures, and not in terms of YAML. (Slaven Rezic)

Comments:

  • This does not mean that I want to replace YAML by a Perl data dump (or maybe yes, but see below). It is just about the specification document: it should not talk about "META.yml", but the META spec, and should not use the YAML data type names (mapping, sequence), but Perl data type names (hash, array).
  • The actual serialization proposals could go into an appendix of the spec, or even in a new document. So whether it will be JSON, or YAML::Tiny, or YAML 1.1 could be decided later.
  • In my opinion I would vote for *two* serialization formats: normal YAML as it is done currently (or even YAML 1.1), which would be used for statistical analysis (here we can force people to install the latest and greatest YAML parser), and a Perl data dump which is used by CPAN.pm etc. for configure_requires or similar stuff (here the user has probably a very bare perl installation, the minimum we can expect is the ability to parse Perl). Something similar is done with distroprefs handling in CPAN.pm: if .yml is available, fine, otherwise it's possible to fallback to .dd, a perl data dump file.
  • I am still fine that the examples in the spec are still expressed in YAML, because I think that YAML is a quite nice human-readable format.

Prerequisites

Enhance granularity of prerequisites

Proposal:

Add "test_requires" distinct from "build_requires". (Adam Kennedy)

This proposal is driven by the needs of the downstream distribution packagers. At Oslo, they requested this separation to allow them to more accurately package modules. I forget, however, the EXACT mechanism that this change was driving downstream, and this MUST be clarified before we add it. Adding a new dependency type simply because it looks like a good idea isn't sufficient, we need to be absolutely sure that it does what the downstream wants before we add it.

Comments:

  • Should we go all the way towards making prerequisites phase-specific and have "configure_requires", "build_requires", "test_requires", and "runtime_requires" (i.e. deprecate the old "requires" entry) (Dagolden) I see no obvious benefit to changing requires -> runtime, just more change, breakages and work (AdamKennedy)
  • Prerequisites are combined in usage during toolchain phases. E.g. "requires" are expected to be available after the configure stage before the build stage. This needs to be clearer in the spec and may have implications for CPAN/CPANPLUS. (Dagolden)
  • Do distribution packages really need test_requires? I cannot think of a use case. (SlavenRezic)
    • They generally don't. That's the reason to have it: test_requires would be a list of modules the CPAN version needs for testing, but downstream could filter out. (JosBoumans)
    • Source Based distros can use it. The users can elect whether or not to perform test phase. Most chose not to, but for those who want assurance, its handy.(kentnl)
    • Speaking from a Debian perspective, whether it is required for testing or for building, we include it as a Build dependency. We try to run as many tests as possible (including author tests, though we prefer AUTOMATED_TESTING to RELEASE_TESTING now). We would really consider test_requires and build_requires equivalent. (jawnsy)

Extensibly Group Prereqs

Proposal:

Rather than have 'config_requires' and 'install_requires' and 'signature_requires' and 'build_recommends', have a two-level system. This requires a small bit of reworking now, but is easy to extend later without adding many top-level keys. Right now (phase)_(verb) is multiplicitive. Both phases (test, install, configure) and verbs (requires, recommends, conflicts) can be added easily over time.

Comments:

  • I would also like to see this split better. It recently came up that a 'recommends' list is inappropriate to list optional testing modules, however as a two-level system, 'test' and 'recommends' might be a better fit, as 'test_recommends' doesn't exist at the moment. (Barbie)
  • This proposal assumes that the downstream consumers will be able to handle all possible combinations. The implications for the downstream are multiplicative, even if the encoding in META.yml are not.

Clarify intent of 'recommends' and add 'suggests'

Proposal:

The 'recommends' flag is not equivalent to Debian recommends. The intent should be made clear for authors and the toolchain. If the Debian definitions are adopted to better match usage by packagers, a 'suggests' field should be added as well. (Adam Kennedy)

Comments:

The current recommends field is ignored, which makes it actually work like "suggests".

If we redefine the actual meaning to match Debian, recommends changes to mean "install unless you say otherwise". This would result in a massive install dependency explosion in most cases. (AdamKennedy)

  • 'recommends' or 'suggests' they both imply (to me) they list optional dependencies. Perhaps we could have both, where 'suggests' implies the toolchain should ask whether the user wishes to install listed dependencies, and 'recommends' simply remains a data-point of dependencies that can be used by the distribution, but are not necessary for its operation after installation. (Barbie)
    • That would be opposite of Debian, with our 'recommends' more like their 'enhances' (Dagolden)
    • Most of the time, the user has less of an ability to answer any "do you want feature blah" questions than the author is. "Asking the user" about anything implies perfect knowledge on the part of the user, as they will need to know the answer not only for the module they are trying to install, but know the answer for all domains of knowledge in all their dependencies recursively -- AdamK
    • In Debian, suggestions are handled by just showing users what packages are suggested. It doesn't do any sort of prompting, though it does give savvy users the chance to install those packages as well. (jawnsy)
  • The "install unless you say otherwise" of Debian is configurable. We consider Recommends to really mean, "The module you're installing is usually found with (recommended modules)" -- though if one wants to be minimalistic they can exclude them. This is precisely our rationale for defaulting to installing them even though the user didn't ask for it. One should ask themselves: "will users consider it strange if X is installed without Y?" (jawnsy)

Add a free-text prerequisite field

Proposal:

Add free-text entries that *describe* prerequisites that cannot be described right now, like "prerequisite: a working libexpat", "Need a perl with defined-or", "Need root access to /media/test". (Tux)

Comments:

This form of prerequisite assumes a human in the loop. In the general installation case, there is no human in the loop. This would also have implications for translation and cross-platform support. Finally, I worry that we'll start ending up with things that could be done more accurately (like the defined-or) or act as an enabler for stupid/weird dependencies "You must be awesome to use this software". (AdamKennedy).

OS/arch/platform-specific requirements

Proposal:

I would like to see a way to specify OS-specific requirements. One option might be to use Devel::CheckOS, which already seems to have a comprehensive list of supported OS names, and have a hash like:

os_requires:
  Linux:
    Linux::Hax: 0
  FreeBSD:
    FreeBSD::Voodoo: 1.2
    BSD::Stuff: 0.999

Alternatively, it could just look for keys matching $^O. Pioto 21:12, 27 August 2009 (BST)

Comments:

Does this add any usefulness? What automation behaviour would it drive? This would also be impossible to auto-generate using current code-based methods in Makefile.PL and would require additional work to produce the static hash. Also, how are junctions covered? File::HomeDir has one dep for Win32 and cygwin, one for Mac, and a different one for all 30 other platforms. AdamKennedy

Hrm. For the "all 30 other platforms" case, I'd say it there should be a special case name, like "default", which is used if no other key matches. Yes, it would likely require a new config paramater for the various build systems. I figure it would basically be a straight passthrough of a hash like the above. Admittedly, for my own use case for this (Unix::Uptime), I don't need special deps on all platforms, just on some of them.

Another possible answer... instead of doing a simple if $conf->{os_requires}{$^O}, do something like foreach (keys %{$conf->{os_requires}}) { if ($^O =~ /$_/) ... } Pioto 12:24, 28 August 2009 (BST)

I think this is going to rapidly approach taking the place of dynamic configuration, and in a bad way. Save a complete proposal for META 3.0 rjbs 15:31, 28 August 2009 (BST)

Allow Sequences (Arrays) for Prereqs

Proposal:

Right now, to the best of my knowledge, the only benefit of a the magic "Bundle::" namespace is that its prerequisites can be declared in optimal installation order. If META allowed prereqs to be communicated in a set order by using a sequence of pairs, Bundle's magic could probably be made obsolete, simplifying the toolchain overall (over a long deprecation period).

Comments:

I quite like the idea of (perhaps optional) ordering of dependencies, but I worry a little that we might come to depend on the installation order, when we really should not. Adam K

Add a post_depends set

Proposal:

Permit specifying of packages that should be installed to provide part of a packages functionality, but should be installed/built *after* the package is installed. (kentnl)

Comments:

  • Being able to specify package that should be installed after the current package is an immeasurably handy tool in dependency resolution. Its especially handy for case of circular dependencies, where the A requires B at runtime, but B requires A at build time. (kentnl)
  • I acknowledge the name above is a crappy one, but the concept is what this proposal matters about. Alternatives welcomed. (kentnl)
  • My main concern with this is that the current model creates a simple directed graph amenable to recursion, and any circular dependencies imposed become rather obvious. I worry that post_requires dependencies will make some of the recursion algorithms too complex to be reliable. I can't prove this yet though, so this is worth keeping an eye and considering in more depth. --Adam K

Prerequisites should be mutually exclusive

Proposal: Modules should only be listed once across all prerequisite categories. E.g. a 'requires' module shouldn't be listed in 'test_requires'; a 'configure_requires' module shouldn't be listed in 'build_requires'. Instead, the spec should define which combinations of categories should be available at which stages of the install process. (Dagolden)

The exception would be 'conflicts', which by its nature should be a duplicate of a module listed elsewhere in the prerequisite structure.

Comments:

  • This would be less work for users and less prone to stupid processing errors -- like generating a list of all requirements and then subtracting 'build_requires' (Dagolden)
  • I think this would remove a certain amount of useful flexibility, standard light weight "META Object" modules could easily automate the production of the merged list. This feels like sacrificing on a fundamental point to make life easier when you don't have tools that haven't been written yet --Adam K

Add development_requires

Proposal: The development_requires field should specify all prerequisites only needed during development. For example this could include templating or other preprocessing modules needed to generate the final source. (Slaven Rezic)

Comments:

  • This would be just convenience for developers, not for the end users. But it happens to be more and more popular to either take over other people's modules, or to fork distributions directly from repositories for patching, so it would be nice to have a concise notation to specify also the developer's prereqs.
  • So one can just write
   cpan `perl -MYAML::Syck -e '$meta = LoadFile shift; for (qw(configure_requires requires development_requires ...)) { print join "", map { "$_\n" } keys %{ $meta->{$_} || {} } } ' META.yml`

and has everything he needs.

Change/clarify existing fields

Better formalization of license field

Proposal:

Replace the list of strings for the "license" field with something extensible and unambiguous. (RicardoSignes)

Comments:

  • I believe Module::Build already allows the use of Software::License classes here. That would be a simple solution.
  • I wouldn't want to depend on an external module for a list. I'd be OK giving an explicit list in the spec drawn from there and saying that lower case names are reserved and anything else must start with a capital letter. Consumers can try to lower case the field and see if Software::License supports it. (Dagolden)
  • When it comes to extensible and unambiguous identifiers, all roads eventually lead to URLs... (AdamKennedy);
  • There won't need to be a list, apart from perhaps 02packages. also, cpan:///package/Software::License::Perl_5 works; it makes it easy to re-use your own license in a machine-readable way by publishing it as a package, and is namespaced and indexed; if it can't be found, treat as Other rjbs 15:21, 28 August 2009 (BST)
  • Right now, the only dual license you can specify is "perl". What if someone wants a dual Artistic 2/GPL 3 license? The current spec says this can only be a string; why not an array? Elliot Shank
  • Licences can differ per-file - eg, code might be covered by Artistic1/GPL2, and doco by something CCish. Or some code might be Artistic1/GPL2 but one file borrowed from another project might be GPL2+. For plenty of distributions, a single entry (whether it be a string or an array) is misleading. David Cantrell
  • ...so we can have Mixed, which is a subset of Other. rjbs 15:21, 28 August 2009 (BST)
  • Devil's advocate position: Does anyone really care about license in META.yml? I mean -- consume it? Any packager or company needs to read the individual files to check for copyright/license statements anyway. How does having an assertion in META.yml help anyone? (Dagolden)
    • Seconded. Perhaps a list of URLs to licences that apply to all or part of the distribution would be more applicable. (Barbie)
    • Terse and automatically encoded forms of the license could be handy for recursive license traversal (for example, to spot clashes in licensing). It also makes life easier for things like search.cpan.org and other websites --Adam K

Make dynamic_config mandatory

Proposal:

Make the 'dynamic_config' a mandatory field (Dagolden)

Comments:

  • dynamic_config is confusing because if missing, it defaults to 1. Further, the default means that META.yml prerequisites are not definitive and configuration must be run. Making this mandatory will at make this confusing flag more obviously apparent and avoid usage errors when missing. (Dagolden)
  • Or rename it to static_config (chorny)

Make repository resource a hash

Proposal:

The repository resource should be made a hash that defines the repository type to disambiguate 'http:' repository urls and clearly indicates the urls for a repository distinct from a repository browser/front-end (#toolchain discussion)

repository:
  type: git
  url: http://git.example.com/some-repo.git
  browser: http://gitweb.example.com/?p=some-repo.git

Comments:

  • Should type be constrained? (Dagolden)
  • What would type actually mean? (AdamKennedy)
  • "type" would indicate which version control system to disambiguate http urls. E.g. an http URL could be a subversion repo or a git repo (which doesn't require a ".git" suffix). (Dagolden)
  • Is it possible that a distribution has multiple repositories? Note that www.ohloh.net allows multiple "enlistments" per project. (SlavenRezic)
  • A proviso for branch-names and purposes ( or just names ) might also wish to be considered, (kentnl) ie:
   repository:
     - type: git
       url: http://git.example.com/some-repo.git
       browser: http://gitweb.example.com/?p=some-repo.git
       purpose: releases
       branch: releases
     - type: git
       url: http://git.example.com/some-repo.git
       browser: http://gitweb.example.com/?p=some-repo.git
       purpose: "patches to"
       branch: master
    • The repository is meant to be a permanent resources and could be listed for years, branches are by definition ephemeral and will go away, recommend against the branches --Adam K

Make bugtracker resource a hash

Proposal:

The bugtracker resource shoudl be made into a hash that defines different types of resources, e.g. web page, email report address, etc. (#toolchain discussion)

Comments:

Formalize optional_features

Proposal:

Optional features: is supported in META.yml, but it requires a lot of manual intervention and trickery to make it work. And it is very poorly documented. (Tux)

Comments:

  • Not all YAML parsers parse that structure the same way, so the order of this section inside META.yml is vital (still :()
  • I for one would be happy to see optional_features: removed entirely, since actual actions resulting from this information only apply in the human-in-the-loop cases. It also locks downstream packagers into either supporting it or not supporting it, none of THEIR users actually get a choice (AdamKennedy).
  • optional_features may be used automatically. This is already implemented in CPAN.pm in conjunction with the "features" key of distroprefs. (SlavenRezic)
  • optional_features can be however quite practical to people with source based distributions as long as they're done properly. Gentoo tends to match each optional_feature to a corresponding 'use' flag. ( Set it and forget it ). (kentnl)
  • The example of optional_features express a optional 'feature' as having 'requires' prerequisites as an 'alternative' to specifying 'recommends'. But what would 'build_requires' within an 'optional_features' entry mean? Is 'recommends' allowed within 'optional_features'? What should CPAN(PLUS) do for each of the prerequisite types if found under an optional_feature entry? (Dagolden)

Clarify author field

Proposal:

I would like to see some clarification of what an author is, especially since the numbers of distros

  • that change ownership
  • that use collaboration
  • that use shared maintenance
  • that cycle releasers frequently

is rising.

  1. Is it the original author (PAUSE jargon »first-come«)?
  2. Is it someone who contributed code?
  3. Is it the original author even though he informally abandoned the distro?
  4. Is it the original author even though he formally transferred or gave up ownership?
  5. Is it a maintainer (PAUSE jargon »co-maint«)? (DBIx-Class has 20 maintainers, and apparently this maintainer information is generally not available in any other way.)
  6. Is it the one maintainer of the current maintainers of the distro who released this distro? (A specific distro can only have exactly one releaser, e.g. RIBASUSHI/DBIx-Class-0.08109.tar.gz.)

Comments:

  • Should the author field be split up into finer grained roles? I don't think so, but keep in mind that the author field is automatically populated by build tools from the Pod of a module, and it should DTRT with respect to what the author field meant/means. (Funny bug: remember when Alias took over Text-Balanced?)
  • I concur on the need for improved meaning. Since META.yml is aimed at machine to machine and is not supposed to be read by humans normally, something like maintainer or release manager or the uploader is potentially the best option here. Assuming we can also lock it down as an email address, having it defined as "who to spam for problems with this module" might be useful AdamKennedy
  • To stay backward compatible and require no toolchain changes, I propose this Pod as a good practice. -- Daxim
  • From inside a distribution directory, there is no way to know who the uploader was or where it lives on CPAN. Should 'author' be the CPANID of the person releasing it? I'm not convinced it should, but want to put it out there for consideration. (Yes, it means changing how things are auto-populated today.) (Dagolden)
  • I think there are two things you want to know about the distribution: who to contact with questions or bugs (in the event that there is no bugtracker) and who holds the copyright. Copyright gets complicated quickly, and is probably best described to humans apart from the case in which a single "license" entry works. Author is probably best as "contact point." One or more people to contact makes sense, and I see no real reason to distinguish between them. rjbs
  • I feel the 'author' field should contain a list of the contributors to the code within. Perhaps an additional field ('maintainer'?) could be added to the mix, using a PAUSEID to identify the release manager/current maintainer. (Barbie)
    • What would a list of contributors actually be consumed by, and what functionality would it drive? The "contact" definition can trigger various "Email the author" type behaviours, but I don't see what useful features a list of contributors might drive beyond the list in the POD --Adam K
  • Picking up on rjbs' point -- perhaps we should follow Perl 6 on this and rename the field to 'auth', short for either 'author' or 'authority' and clarify that it's a contact person, not a copyright holder (Dagolden)

Add new fields

Have a "development version" flag

Proposal:

Add a boolean value indicating whether the release is a "production" or "alpha/developer/beta/whatever" release. Elliot Shank

Comments:

  • Pro: We'd have an unambiguous way of specifying that something is or is not "production ready", without any silly parsing of version numbers that include or don't include underscores.
  • Con: I can't see the current system for determining this being deprecated, so we'd have two systems to support. Elliot Shank
  • Con: Development version'ness would not be determinable post-installation AdamKennedy

Add a "release_status" field

Proposal:

  • Add a mandatory release_status field with a limited list of acceptable values. E.g. "stable", "testing" and "unstable", with "alpha" as an alias for "unstable" and "beta" as an alias for "testing".
  • Have PAUSE use this field first to determine whether distributions should be indexed. (Eventually, have PAUSE create separate indexes "stable", "stable+testing", "stable+testing+unstable".)

Comments:

  • This is more useful than the boolean 'development version' proposal as different authors use development versions for both testing and unstable. In response to Adam's concern about knowing status post-installation, see the proposal to install Meta files and make them queryable. (Dagolden)
  • Should "release candidate" be an allowed value? Should it be an alias for "testing" or should it exist on its own? (Dagolden)
    • "release candidate" is mostly just a hint to humans that this is your last chance for testing... I don't see any difference in behaviour for consumers at this point --Adam K

Controlling test suite runs

Proposal:

The META spec should add flags to indicate that distribution's tests may be shuffled or run in parallel. (SlavenRezic)

Comments:

The prove utility provides the interesting options --shuffle and --jobs. With --shuffle the test suite is not run in alphabetic but random order. With --jobs multiple test scripts run in parallel. This could speed up testing, especially on multi-processor machines.

Unfortunately authors may write their test suites that neither one or the other option may be used: authors may have some "initialization" and "cleanup" script in the test suite (so shuffle cannot be used), or may use a file or other resource in many test scripts (so parallel runs cannot be used). So we cannot assume that --shuffle and --jobs may be used by default.

A sample definition could look like this (please think of better key names):

test_run:
  may_be_shuffled: yes
  may_run_in_parallel: yes

Maybe even a control file to manage chunks/blocks that the module authors already set up (Tux).

  test_conf_file: t/t_conf
  • I quite like this idea in principle, mostly because I've never seen a better suggestion for how the Makefile.PL or Build.PL could make parallel testing work. That said, it seems like it SHOULD be possible to make this happen without having to go via metadata (which is for the configuration to communicate to the CPAN client). Could the CPAN client really do anything about this? --Adam K

Specify a DLSIP resource

Proposal:

DLSIP codes should be specified in META.* as a resource.

Comments:

search.cpan.org has a URL that resolves and describes the DLSIP coding. I am suggesting that a resource be defined that uses this url (or others like it) in order to specify the dlsip code for the module. (For automated processing, we could specify that the dlsip code would be after the last non-alphabetic character.) (CSJewell)

A sample use would look like this:

 resources:
   dlsip: http://search.cpan.org/dlsip?apdOp
  • Personally I find the URL redundant. What if it ever changes? Also 'resources' feels to be the wrong place to list this, as it's an attribute of the distribution not an external resource. However, I would like to see this listed somewhere, as it can be a useful data point about the modules within the distribution. (Barbie)

Long description

Proposal:

Field for long description that can be used in search, when generating README, OS packages or submitting dist to Ohloh, Freshmeat and similar sites. --Chorny 19:40, 21 September 2009 (BST)

Comments:


Short description

Proposal:

Field for short description that can be used in search (with higher priority than long description), when generating README, OS packages or submitting dist to Ohloh, Freshmeat and similar sites. 'abstract' currently is module-specific (for ex. for Shipwright: "Best Practical Builder") and would not tell much about module purpose in general. --Chorny 01:15, 28 September 2009 (BST)

Comments:


Language

Proposal:

Perl 6 is coming. Some code in Perl 6 is already being uploaded to the CPAN. A new "language" field is an important part of the structure we need to allow Perl 6 to reuse the existing CPAN rather than to try to reinvent the whole thing. My recommendation would be that this be an optional field with an implicit value of 'perl5' --Jesse

Comments:

  • At this time, META.yml is specifically oriented around Perl 5. To suppose multiple languages, much more than just a language flag would need to be changed. --Adam K

Trove-Like Categories

Proposal:

Add Freshmeat.net or SourceForge-like trove categories with topics programming languages, etc. As opposed to keywords/labels/tags they are a nested tree, and will also be limited to a given list. This will help in being able to better browse CPAN instead of the old and heavily under-maintained long modules list. For example "Programming Language :: Lisp", "Intended Audience :: Emacs Users", "Operating System :: GNU", "Topic :: Editors".

Shlomi Fish wrote a functional spec for the proposal of enhancing the CPAN classification which may be read for further enlightenment.

Comments:

Freshmeat no longer has Trove categories, they were replaced by tags. --Chorny 15:40, 27 September 2009 (BST)

Version changes

Proposal:

Description of changes in that versions and tags for changes. Useful for submission to Freshmeat and for quick review of changes. --Chorny 18:51, 30 September 2009 (BST)


Comments:

Other (not yet classified)

steal from other package meta specs

Proposal:

Hunt down the spec authors and some heavy package users, ask what in the spec/package meta files works for them and what doesn't, then steal the best features. -- Daxim

Comments:

Install Meta files and make them queryable

Proposal: Using something like .packlist or File::ShareDir, the Meta file should be installed along with the distribution and Meta tools should allow easy querying of meta information for installed modules. (Dagolden)

Comments:

  • Lots of meta information is unavailable in a structured way after a distribution is installed. By installing the Meta file (e.g. in auto/meta/Foo-Bar/META.yml) we retain it after installation and can make it available for other tools. (Dagolden)
  • Should the installed meta be identical to the Meta file in the distribution or should it be stored in a neutral format (e.g Data::Dumper)? (Dagolden)
  • I'd be happy to just install the MYMETA.yml file as is. Perhaps it could be extended slightly to create a "packlist 2.0" type file, it's been a long time since packlists have been reviewed. -- Adam K

Binary Package Dependencies

Proposal:

Add a binary dependency keyword to be optionally resolved by the shell. eg.

 binary_requires:
   - linux-debian:
     libgif4-dev: 4.1.6
   - linux-ubuntu:
     libgif4-dev: 4.1.6
   - linux-redhat:
     giflib-devel: 4.1.6
   - freebsd-ports:
     giflib: 4.1.6
   - freebsd-pkg:
     giflib4: 4.1.6

The aim is for use by Alien:: namespace modules to declare a dependency that would allow them to detect their desired non-Perl dependency without installing a copy of the code involved that isn't tracked by the system package manager (and hence isn't subject to security updates.

I don't expect immediate implementation by CPAN/CPANPLUS, but it provides a structure where extensions to either can find the information needed to install those packages if the user desires.

If an Alien::* module's configuration (Makefile.PL/Build.PL) were started without the packages installed, then either the user chose not to install the packages, or no package was found for the given platform. The Alien::* could then install from source (or fail.)

A Makefile.PL or Build.PL doesn't necessarily have the information needed to know whether to install a package, while CPAN/CPANPLUS may, especially given the possibility of extra configuration (o conf binary_requires on|off).

One weakness with the above is the lack of alternatives, a few years ago we might have been satified by either libgif or libungif (Tonyc)

Comments:

  • The biggest challenge with this proposal is the need for a target-agnostic specifier. If a truly portable method exists for defining "libgif version 4.1.6 or better" then you don't need to say which packages are needed, as the resolution of a package from the arbitrary identifier can be solved independently. The need for massive amounts of consulting with other communities to achieve some useful consensus on identifiers is the biggest challenge for this proposal. --Adam K

Make repository more machine readable

Proposal

Currently we have a simple URL for the repository contained in the resources/repository key. It would be good if more information could be specified in order to allowed automated tools to do manipulations on the repository rather than the shipped distribution

I suggest that we *optionally* allow a hash of data to be stored in repository:

  • A distinct URL for the web front end to the repsotitory and the repository itself
  • The repository format (cvs, svn, darcs, git, etc)
  • The repository type (free form string, but I'm thinking "sourceforce" "github" etc, that would allow automated tools that know how to use repositories that function like those webhosts to do the right thing

For example:

 resources:
    repository:
       web: http://github.com/2shortplanks/test-time-hires
       url: git://github.com/2shortplanks/test-time-hires.git
       format: git
       type: github


Comments:

After thinking about this for a while I wonder if we should keep the use of repostiory as it is now and add a second key called "version-control" that has the broken down info. This would make our META.yml more verbose, but would mean that it was backwards compatible with clients that only know how to read plain strings in repository

Formally reserve keys for private 'in-house' use

Proposal:

For in-house, non-CPAN use, it would be nice to be able to extend META.yml to contain extra keys and data, but any extensions that we choose to make may one day clash with an official key not yet introduced.

For this reason I'd like to formally define the acceptance of 'x-' prefixed key names. Clients that validate META.yml should ignore any and all keys starting with 'x-' and those keys should be formally declared as never being used in future extensions to the spec.

Comments:

  • This seems like a fairly good way to allow experimentation with new features without changing the spec, and pretty non-controvertial in general --Adam K
  • I thought this was what key names containing a capital letter did? CSJewell

Other brainstorming (not full proposals)

This is a laundry list of ideas I've picked up from people over the years. If people actually like them, please flesh them out and add them to "Suggestions" above. Or add new ideas as a placeholder for others to expand on. (Dagolden)

  • Allow specifying dependencies on a particular tarball (e.g. an alpha version)
  • Add a 'support' resource as a hash with 'email', 'forum', 'website', etc keys
  • Rewrite more formally using 'must', 'may', 'must not' and so on to clarify usage in toolchain modules
  • Add notes in spec about META consumer behavior (e.g. degrade gracefully, error conditions, dealing with newer/older specs, etc.)
  • Make Unicode support explicit (whether allowed or prohibited)
  • Optional JSON support
  • Add a way to specify requirement 'junctions' (e.g. at least one of Foo::PP, Foo::XS or Foo::LibFoo)
  • Add explicit validation criteria to all spec entries
  • Add DSLIP information (or equivalent) to META
Personal tools