Challenge Setting Issues

List of different issues caused by using the challenge setting process for defining funding categorisations

Challenge setting process issues

High voter effort

New funding categorisations in every funding round increases the effort and complexity for voters to participate. Voters needs to spend time reviewing all the challenges and take into account the focus areas and budget of each challenge and the most pressing needs of the ecosystem before they can make an informed decision. As the number of challenges increases there is increasing effort needed which decreases the number of voters who can participate due to the time requirements to vote. This could lead to increasing centralisation if an increasing number of voters are delegating their vote to another representative or simply not participating at all.

High proposal assessor effort

Constantly changing funding categorisations in each funding round results in a high effort required from proposal assessors to assess the challenges submitted each time and for those assessments to be reviewed. This uses up funding resources and time from assessors that could be spent assessing other idea or contributor based proposals.

High governance complexity

The challenge setting process results in suggesting new funding categorisations every funding round with different focus areas and budget weighting. This results in a recurring and complex governance decision for the community. The community needs to be well informed on the needs of the ecosystem to make well informed decisions on which categorisations to include and exclude. This is even more important when those decisions could mean preventing certain ideas from being able to submit proposals. The communities confidence would need to be higher to know that the excluded focus areas would not have proposals submitted that were more impactful than the ones that could be submitted.

Expensive process due to incentives used for challenge teams, assessments and voting

Challenge setting is an expensive process as it requires incentivised input from challenge team proposers, proposal assessors, veteran proposal assessors and voters in every single funding round. These stakeholders also need to have an understanding of the wider ecosystem to make important categorisation decisions.

Challenging to scale an increasing number of challenges

Challenge settings become more complex as they scale as if the community size increases it will likely lead to an increasing amount of challenge proposals, assessments and time needed for voters to assess and vote on those challenges. A larger amount of challenges increases the complexity of deciding between challenges making it a time and resource expensive process to scale. When coupled with an increasing amount of idea based proposals the funding process becomes increasingly complex and time consuming for the average voter to participate.

High proposer effort

Proposers in every funding round have to read the challenge settings that were selected to work out whether they can submit proposals. Constantly changing funding categorisations means this effort is needed every funding round. If a funding round is exclusionary to certain ideas then the proposer must interpret whether they can or can't submit certain ideas. If the categorisations are overlapping then the proposer must spend time comparing challenges to determine where best to submit their proposal.

Challenge setting proposal issues

Limited challenge team knowledge or ecosystem understanding

Currently anyone can create a challenge setting proposal. This increases the risk that proposal teams are not fully informed about the ecosystem and won't be as effective in creating challenge settings that detail important areas with sensible budget weightings.

Lack of comprehensive data or supporting evidence to justify categorisations

Currently proposers do not need to provide any data or supporting evidence as to why the challenge setting is needed at this precise time in the ecosystem over other potential focus areas. There is also no incentive for them to take into consideration the needs of other focus areas of the ecosystem and take that into account when justifying their own categorisation and budgets. This increases the risks that the challenge setting may not be as well justified or not even be a high priority for the ecosystem over other suggested funding categorisations.

Low barrier of entry for malicious actors

Anyone can create challenge setting proposals meaning there is a very low barrier for anyone to try and harm the process. Malicious actors could for example spam the system, try to persuade the community to go down a less impactful path or add duplicate challenge proposals with slightly different topics or budgets to increase overall complexity.

Challenges influenced by self interest

Currently anyone can create a challenge setting proposal which opens up the system for people, teams or companies creating challenge settings to help serve their self interest instead of the overall needs of the community. If this incentive exists it should be assumed that it will be abused without the right processes and protections in place to prevent it. Adding those moderation efforts into the process would further increase the cost and resources needed to use challenge settings.

Lack of categorisation moderation

The assessment stage is currently the only process that moderates the content of funding categorisation and does not help with merging and updating categorisations. Adding in a moderation process to resolve this can be achieved however it would add yet another cost to the usage of challenge settings.

Unilateral budget weighting decisions

The budget weighting for challenge settings is self determined by the challenge team. The proposer does not need to follow any process, provide data or use any democratic process to determine the budget used for a challenge setting. Budget weighting decisions with challenge settings are unilateral and not democratic.

Common categorisation issues

Complexity justifying specific challenge settings

The general guidelines for challenge setting proposals are to not be too broad but also not too specific. As the proposer is free to make any challenge the risk still exists that the challenge could be overly specific. For instance a challenge setting could focus on local events in Japan. The population for Japan is around 125 million. The total worldwide population is 7.9 billion. The challenge would target around 1.58% of the global population which makes the challenge very specific. This adds complexity as more data, reasoning and evidence is needed to provide confidence on whether this specificity of focus is an effective way to direct funding over other options. The example above is not trying to suggest what specific challenge is and is not effective for funding allocation. Instead the only important thing to highlight is how specific funding categorisations increase the complexity of justifying which categorisations to include and exclude and the governance complexity of that decision. This complexity is a time and resource cost that impacts all stakeholders in the ecosystem.

Budget weighting complexities for specific challenge settings

The more specific the challenge settings are the more complexity there is in determining what an effective budget weighting should be. Funding categorisation is predictive which means challenge teams will have to predict what budget is sensible for that specific focus area and then also take into account how many proposals might be submitted. Increasing the number of specific funding categorisations increase the complexity of applying effective budget weightings as there is an increased chance of being wrong in the prediction of what proposals and their budget requirements are actually submitted.

Challenge settings struggle to handle adverse situations

Challenge settings invite any form of categorisation. Specific categorisations easily lead to situations that lead to bad outcome. A given goal, objective or specific focus area can have situations where few proposals turn up or that ones that do are low quality. There are also situations where something in the community becomes higher priority or an existing goal or objective becomes lower priority. Specific categorisations caused by the challenge setting process aren't flexible to a dynamically changing environment and requirements.

Specific challenge settings or exclusive funding rounds increases the risks of weaker competition

Using an increasing number of specific funding categorisations leads to reducing the competition between proposals as only certain proposals can be submitted that address that specific focus area. This reduction in competition can lead to easier access to funding for weaker proposals as voters would be unable to direct the funding to other focus areas with higher quality proposals. When focus areas are excluded from a funding round this also reduces competition as that means there is more funding available for a smaller range of idea meaning there is more funding between fewer ideas.

No funding access for certain ideas and innovation

Challenge settings compete with one another every funding round. This makes it easier for focus areas with different ideas and innovation to be excluded from the funding process depending on the voting outcome. This means potentially impactful proposals cannot request funding. This leads to stifling innovation and progress in the ecosystem for certain focus areas. An exclusionary funding categorisation environment can also easily result in excluding minorities of the ecosystem. Minorities that have less voting power will struggle to promote and get as much success in getting categorisations that are relevant to their own goals and objectives to be voted on and selected by the community.

Duplicated or overlapping challenge settings

There is no process currently to prevent duplicate challenge settings or challenges that are similar to other challenges. This creates the risk of having similar challenges funded that result in a far greater amount of funding being directed to only a few areas of the ecosystem. This can cause a stagnation in innovation and community efforts for areas that need funding support. Overlapping categorisations also make it more difficult to analyse how funding is actually being directed to certain focus areas.

Difficult to effectively direct funding

Allowing anyone to submit challenge settings with any focus area and budget leads to complex voting decisions for the voters. This decision complexity causes the issue of making it more difficult for the community to effectively direct funding to multiple focus areas in the ecosystem due to the large amount of competing and potentially overlapping categorisations.

Last updated