Changes between Initial Version and Version 1 of CreditGeneralized

Timestamp:

Aug 8, 2014, 5:48:53 PM (10 years ago)

Author:

davea

Comment:

--

CreditGeneralized

@@ +1 @@
+= Generalizing credit =
+
+Note: in the following
+ * FLOP = floating-point operation
+ * FLOPs = plural of FLOP
+ * FLOPS = FLOPs per second
+
+The current credit system is based on FLOPs:
+a 1 GFLOPS computer running BOINC all the time gets 200 credits per day.
+We did things this way because BOINC was designed to support
+scientific computing, where most apps are floating-point intensive
+and FLOPS is the standard unit of performance
+(for example, supercomputer performance is measured in FLOPS).
+
+For grant-writing purposes I need to be able say
+"BOINC has a peak performance of X PetaFLOPS";
+the current credit system lets me do this.
+
+Recently, a new project (Bitcoin Utopia, or BU) started.
+Their jobs involve Bitcoin mining,
+which consists of computing SHA256 hash functions.
+This is an integer algorithm.
+It can be done on CPUs and GPUs, but it can be done much faster on ASICs.
+These ASICs can only do hashing; they can't do floating-point math,
+and they can't be used by any BOINC project other than BU.
+
+The question was: how should BU grant credit for its jobs?
+One approach is to decide on an "equivalence" between hashes and FLOPs,
+and assign credit based on the current formula.
+How many FLOPs is a hash equivalent to?
+One approach is to look at a CPU or GPU, measure its FLOPS and its hashes/sec, and divide.
+Depending on the device, this gives an answer in the range of 1,000 to 10,000 FLOPs per hash.
+
+BU did more or less these things, which are completely reasonable.
+But it turns out - because ASICs are so fast -
+that BU is granting huge amounts of credit.
+With less than 1,000 users, BU is granting more credit than the 300,000 users of all other projects combined.
+
+This situation has 2 undesirable consequences:
+
+ * Credit no longer measures FLOPs; BOINC's combined average credit
+   no longer measures its peak performance in FLOPS.
+ * The competitive balance between projects is lost.
+   BU will always grant far more credit than other projects,
+   for a type of computation that is specific to BU
+   and is not usable for other projects.
+
+== Proposal ==
+
+The basic problem is that we have a credit system based on FLOPs,
+but we want to give credit for things (like hashes) that are not FLOPs.
+A similar situation actually already exists in BOINC.
+We'd like to be able to give credit for disk storage and network communication;
+some projects have applications that use these resources rather than computing.
+But there's no obvious way to translate storage or bandwidth into "equivalent FLOPs",
+and even if there were, we'd be destroying the meaning of credit as a measure of FLOPs.
+
+So, I propose that, rather than trying to shoehorn everything into one number,
+we keep track of multiple types of credit.
+In particular, I propose 4 types:
+
+ * Computing credit: general-purpose FLOPs, i.e. what we have now.
+ * Disk storage, measured in byte/seconds.
+ * Network throughput, measured in bytes, the sum of upload and download.
+ * Project-specific credit.
+   Projects can define and grant this however they like.
+   For BU, this would be hashes.
+   Other projects, like Wildlife@home, might grant credit for
+   a human activity like annotating video.
+
+The BOINC database would maintain each of these types of credit
+for each host, user, and team.
+It would store both total and recent average for each type.
+
+The new types of credit would be included in
+the XML statistics files exported by projects.
+Statistics sites (such as BOINCStats) could be extended to
+show the new types of credit.
+
+== Discussion ==
+
+Eric pointed out the possibility of a variant of the SETI@home app
+that uses an ASIC or FPGA to compute FFTs.
+What if these were 1000X faster than GPUs or CPUs?
+We'd have the same problem as we do now with BU.
+
+My feeling about this is that computing credit should be measure
+'general-purpose' FLOPs, i.e. FLOPs that are usable by most science applications.
+FFT FLOPs are not general-purpose.
+So the right thing would be for SETI@home to grant both computing credit
+and project-specific credit.
+CPU and GPU jobs would be granted both;
+jobs done by ASICs or FPGAs would be granted only project-specific credit.
+
+Similarly, BU could grant computing credit for mining jobs done by CPU or GPU;
+but for ASIC jobs it would grant only project-specific credit.
+
+Of course this is all subjective and fuzzy;
+you might argue that GPU FLOPs are not general-purpose because
+some apps don't map well to GPUs.
+But we need to draw a line somewhere,
+and I think we've reached a point where GPUs can be considered general-purpose.