Reduced Verifiability

Considerations for how the community will be able to verify and audit the impact of proposals based on funding categorisation

Problem

Categorisations that don't add specific metrics and KPIs for the types of proposals that can be submitted makes it more difficult to verify the outcome of the proposals against the funding categorisation they were submitted in.

Issues with current approach

There are issues with the existing approach of applying metrics to challenges that should be explored before considering how we can verify whether metrics and objectives are being met. The current approach for challenge setting includes the challenge teams setting some metrics for them to be able to verify whether the challenge has produced a good return on intention.

The major issue with putting metrics in challenges is it is very difficult to get coverage of all the types of metrics that could be useful for the proposals that are submitted. For instance, for DeFi there could be a number of metrics such as TVL, transaction volume, transaction size, number of wallets, number of actual users etc. If a challenge team was to only pick a few metrics and say these are the ones that must be tracked then that could mean being exclusionary to innovation that does not match well with focussing on those metrics. If the challenge team tries to include all forms of metrics then this adds more complexity on determining whether the challenge is verifiable. Proposals that are submitted can be very different in what they want to achieve and also could be at different stages of development. All this considered, trying to mandate a single set of metrics for all proposals is both not easy nor an effective approach. Goals and objectives with metrics do not need to be attached and embedded into funding categorisations.

Relevance to funding categorisation

Attaching metrics and KPIs to funding categorisations is a choice and is not required. Goals and objectives with metrics and KPIs to achieve can be set and used in a completely independent process. None of the funding categorisation approaches have any direct impact on whether or a proposal or group of proposals is auditable or not. Proposals should detail out how their outcome will be measured so that this can be considered by the voters and then be audited by the community if they are funded.

Potential Solutions

Proposal focussed metrics

Proposals that go through the funding process benefit from being able to present what metrics and KPIs are relevant to their proposal that will be used upon completion of the proposal to assess whether it had a good return on intention or not. To support proposals being able to effectively articulate their KPIs and metrics a comprehensive number of examples could be created for different proposal types. These examples could help to improve the overall quality of metrics produced by proposal teams and make it easier to track whether they had an overall positive impact. Proposals are the best place to have tailored and specific metrics added that apply to that project for the community to give feedback to on whether they make sense and could lead to good outcomes.

Community priorities

Goals and objectives can be set by the community where a high level goal of what they want to achieve and specific objectives could be shared and voted on. A goal could be to 'Increase the number of open source identity wallets and infrastructure'. In response to that goal an identity focussed proposal could add in specific metrics for an open source identity wallet to track how many people start using it or how many public issuers and credential holders there are. Goals and objectives set by the community could be integrated into the proposal submission process so proposers can state which goals and objectives they will help with tackling. Proposers would be able to add more specific details on the metrics and KPIs relevant to their proposal that help to achieve those goals and objectives.

Completion reports & community feedback

Goals and objective are great to steer proposals to achieve certain outcomes however a key part of truly determining whether a funded proposal has achieved these goals and objectives and provided a positive impact is to incorporate some qualitative information from both the proposal team and the community with feedback nearer completion. Experts and topic focussed community groups are a good example of people that could review the results and overall objectives to then provide their informed opinion on whether a good return on intention was achieved. This opinion could come in the form of a full report, a numerical score or a simple review. The better this feedback is for the outcome of the proposal the more informed the community would be on future funding decisions.