1 /* 2 * Licensed to the Apache Software Foundation (ASF) under one 3 * or more contributor license agreements. See the NOTICE file 4 * distributed with this work for additional information 5 * regarding copyright ownership. The ASF licenses this file 6 * to you under the Apache License, Version 2.0 (the 7 * "License"); you may not use this file except in compliance 8 * with the License. You may obtain a copy of the License at 9 * 10 * http://www.apache.org/licenses/LICENSE-2.0 11 * 12 * Unless required by applicable law or agreed to in writing, software 13 * distributed under the License is distributed on an "AS IS" BASIS, 14 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 * See the License for the specific language governing permissions and 16 * limitations under the License. 17 */ 18 19 package org.apache.giraph.ooc.policy; 20 21 import com.sun.management.GarbageCollectionNotificationInfo; 22 import org.apache.giraph.ooc.command.IOCommand; 23 24 /** 25 * Interface for any out-of-core oracle. An out-of-core oracle is the brain of 26 * the out-of-core mechanism, determining/deciding on out-of-core actions (load 27 * or store) that should happen. 28 * Note: any class implementing this interface should have one and only one 29 * constructor taking only two arguments of types 30 * <code>ImmutableClassesGiraphConfiguration</code> and 31 * <code>OutOfCoreEngine</code> 32 */ 33 public interface OutOfCoreOracle { 34 /** 35 * Different types of IO actions that can potentially lead to a more desired 36 * state of computation for out-of-core mechanism. These actions are issued 37 * based on the status of the memory (memory pressure, rate of data transfer 38 * to memory, etc.) 39 */ 40 enum IOAction { 41 /** 42 * Either of: 43 * - storing incoming messages of any partition currently on disk, or 44 * - storing incoming messages' raw data buffer of any partition 45 * currently on disk, or 46 * - storing partitions' raw data buffer for those partitions that are 47 * currently on disk. 48 */ 49 STORE_MESSAGES_AND_BUFFERS, 50 /** 51 * Storing a partition that is *processed* in the current iteration cycle. 52 * This action is also known as "soft store" 53 */ 54 STORE_PROCESSED_PARTITION, 55 /** 56 * Storing a partition from memory on disk, prioritizing to *processed* 57 * partitions on memory. However, if there is no *processed* partition, 58 * store should happen at any cost, even if an *unprocessed* partition has 59 * to be stored. This action is also know as "hard store". 60 */ 61 STORE_PARTITION, 62 /** 63 * Loading an *unprocessed* partition from disk to memory, only if there are 64 * *processed* partitions in memory. This action basically initiates a swap 65 * operation. 66 */ 67 LOAD_TO_SWAP_PARTITION, 68 /** 69 * Loading an *unprocessed* partition from disk to memory. This action is 70 * also known as "soft load". 71 */ 72 LOAD_UNPROCESSED_PARTITION, 73 /** 74 * Loading a partition (prioritizing *unprocessed* over *processed*) from 75 * disk to memory. Loading a *processed* partition to memory is a prefetch 76 * of that partition to be processed in the next superstep. This action is 77 * also known as "hard load". 78 */ 79 LOAD_PARTITION, 80 /** 81 * Loading a partition regardless of the memory situation. An out-of-core 82 * mechanism may use this action to signal IO threads that it is allowed to 83 * load a partition that is specifically requested. 84 */ 85 URGENT_LOAD_PARTITION 86 } 87 88 /** 89 * Get the next set of viable IO actions to help bring memory to a more 90 * desired state. 91 * 92 * @return an array of viable IO actions, sorted from highest priority to 93 * lowest priority 94 */ 95 IOAction[] getNextIOActions(); 96 97 /** 98 * Whether a command is appropriate to bring the memory to a more desired 99 * state. A command is not executed unless it is approved by the oracle. This 100 * method is specially important where there are multiple IO threads 101 * performing IO operations for the out-of-core mechanism. The approval 102 * becomes significantly important to prevent all IO threads from performing 103 * identical command type, if that is a necessity. For instance, execution of 104 * a particular command type by only one thread may bring the memory to a 105 * desired state, and the rest of IO threads may perform other types of 106 * commands. 107 * 108 * @param command the IO command that is about to execute 109 * @return 'true' if the command is approved for execution. 'false' if the 110 * command should not be executed 111 */ 112 boolean approve(IOCommand command); 113 114 /** 115 * Notification of command completion. Oracle may update its status and commit 116 * the changes a command may cause. 117 * 118 * @param command the IO command that is completed 119 */ 120 void commandCompleted(IOCommand command); 121 122 /** 123 * Notification of GC completion. Oracle may take certain decisions based on 124 * GC information (such as amount of time it took, memory it reclaimed, etc.) 125 * 126 * @param gcInfo GC information 127 */ 128 void gcCompleted(GarbageCollectionNotificationInfo gcInfo); 129 130 /** 131 * Called at the beginning of a superstep. 132 */ 133 void startIteration(); 134 }