# Class SpatialTree¶

## Class Documentation¶

class SpatialTree : public virtual necsim::Tree

Represents the output phylogenetic tree, when run on a spatially explicit landscape.

Contains all functions for running simulations, outputting data and calculating coalescence tree structure.

Subclassed by necsim::ProtractedSpatialTree

Public Functions

SpatialTree()

The constructor for SpatialTree.

~SpatialTree()
void runFileChecks()

Runs the basic file existence checks. Checks for paused simulations and file existence.

void checkFolders()

Checks that the folders exist and the files required for the simulation also exist.

void setParameters()

Sets the map object with the correct variables, taking the SimParameters structure defined elsewhere for the parameters.

Requires that parameters have already been imported into the SimParameters

This function can only be run once, otherwise a Main_Exception will be thrown

void importMaps()

Imports the maps into the landscape.

The simulation variables should have already been imported by setParameters(), otherwise a Fatal_Exception will be thrown.

void importActivityMaps()

Imports the activity maps from the relevant files.

unsigned long getInitialCount()

Counts the number of individuals that exist on the spatial grid.

Return
the number of individuals that will be initially simulated

void setupDispersalCoordinator()

Sets up the dispersal coordinator by linking to the correct functions and choosing the appropriate dispersal method.

void setup()

Contains the setup routines for a spatial landscape. It also checks for paused simulations and imports data if necessary from paused files. importMaps() is called for importing the map files.

unsigned long fillObjects(const unsigned long &initial_count)

Fill the active, data and grid objects with the starting lineages.

Return
the number of lineages added (for validation purposes)
Parameters
• initial_count: the number of individuals expected to exist

unsigned long getIndividualsSampled(const long &x, const long &y, const long &x_wrap, const long &y_wrap, const double &current_gen)

Gets the number of individuals to be sampled at the particular point and time. Round the number down to the nearest whole number for numbers of individuals.

Return
the number of individuals to sample at this location.
Parameters
• x: the x location for individuals to be sampled
• y: the y location for individuals to be sampled
• x_wrap: the number of x wraps for the cell
• y_wrap: the number of y wraps for the cell
• current_gen: the current generation timer

unsigned long getNumberLineagesAtLocation(const MapLocation &location) const
unsigned long getNumberIndividualsAtLocation(const MapLocation &location) const
void removeOldPosition(const unsigned long &chosen)

Removes the old position within active by checking any wrapping and removing connections.

The function also corrects the linked list to identify the correct nwrap for every wrapped lineage in that space.

Parameters
• chosen: the desired active reference to remove from the grid.

void calcMove()

Calculate the move, given a start x,y coordinates and wrapping.

The provided parameters will be altered to contain the new values so no record of the old variables remains after function running. Current dispersal methods use a fattailed dispersal.

long double calcMinMax(const unsigned long &current)

Calculates the minmax for a given branch.

Calculates the speciation rate required for speciation to have occured on this branch.

Parameters
• current: the current active reference to perform calculations over.

void calcNewPos()

Calculates the new position, checking whether coalescence has occured and with which lineage.

This involves correct handling of checking wrapped lineages (outside the original grid). The probability of coalescence is also calculated.

void calcWrappedCoalescence(const unsigned long &nwrap)

Calculates the coalescence event when the target cell is wrapped.

Parameters
• nwrap: the number of wrapped lineages

void switchPositions(const unsigned long &chosen)

Switches the chosen position with the endactive position.

Parameters
• chosen: the chosen lineage to switch with endactive.

void calcNextStep()

Calculates the next step for the simulation.

unsigned long estSpecnum()

Estimates the species number from the second largest minimum speciation rate remaining in active.

This allows for halting of the simulation once this threshold has been reached. However, the function is not currently in use as calculating the coalescence tree is very computionally intensive.

void incrementGeneration()

Increments the generation counter and step references, then updates the map for any changes to habitat cover.

void updateStepCoalescenceVariables()

Updates the coalescence variables in the step object.

void recordLineagePosition()

Zeroes out the coalescence information and stores the origin location.

void addLineages(double generation_in)

Expands the map, generating the new lineages where necessary.

The samplemask provided is used for expansion. Any empty spaces are filled with a new lineage. Lineages which have not moved are changed to tips, with a new data entry so that original and new generations are recorded.

Parameters
• generation_in: the generation that the expansion is occuring at. This is used in recording the new tips

string simulationParametersSqlInsertion()

Creates a string containing the SQL insertion statement for the simulation parameters.

Return
string containing the SQL insertion statement

void simPause()

Pause the simulation and dump data from memory.

void dumpMap(shared_ptr<ofstream> out)

Saves the map object to file.

Parameters
• out: the output file stream to save the object to

void dumpGrid(shared_ptr<ofstream> out)

Saves the grid object to file.

Parameters
• out: the output file stream to save the object to

void simResume()

Resumes the simulation from a previous state.

Reads in the parameters and objects from file and re-starts the simulation.

void loadGridSave(shared_ptr<ifstream> in1)

Loads the grid from the save file into memory.

Note
Requires that both the simulation parameters and the maps have already been loaded.

void loadMapSave(shared_ptr<ifstream> in1)

Loads the map from the save file into memory.

Note

void verifyActivityMaps()

Checks that the reproduction map makes sense with the fine density map.

void addWrappedLineage(unsigned long numstart, long x, long y)

Adds the lineage to the correct point in the linked list of active lineages.

Parameters
• numstart: the active position to add
• x: the x position of the lineage
• y: the y position of the lineage

unsigned long countCellExpansion(const long &x, const long &y, const long &xwrap, const long &ywrap, const double &generationin, vector<TreeNode> &data_added)

Counts the number of lineages at a particular location that need to be added, after making the correct proportion of those that already exist into tips.

Return
the number of lineages still to add
Parameters
• x: the x coordinate of the location of interest
• y: the y coordinate of the location of interest
• xwrap: the x wrapping of the location
• ywrap: the y wrapping of the location
• generationin: the generation to assign to new tips
• data_added: vector containing TreeNode objects to add to data

void expandCell(long x, long y, long x_wrap, long y_wrap, double generation_in, unsigned long add, vector<TreeNode> &data_added, vector<DataPoint> &active_added)

Expands the cell at the desired location by adding the supplied number of lineages.

This takes into account wrapping to correctly add the right number

Parameters
• x: the x coordinate to add at
• y: the y coordinate to add at
• x_wrap: the x wrapping to add at
• y_wrap: the y wrapping to add at
• generation_in: the generation to set the new lineages to
• add: the total number of lineages to add at this location

void addGillespie(const double &g_threshold)
bool runSimulationGillespie()
void runGillespieLoop()
void setupGillespie()

Sets up the Gillespie algorithm.

void setupGillespieLineages()

Resets lineage speciation rates to 0.

void setupGillespieMaps()
Cell getCellOfMapLocation(const MapLocation &location)

Calculates the x, y position on the fine map of the lineage.

Return
cell object containing the x, y location
Parameters
• location: the map location

void findLocations()

Finds the locations that lineages are at and adds them to the list of locations.

This also involves calculating the event probabilities for each cell.

void checkMapEvents()
void checkSampleEvents()
void gillespieCellEvent(GillespieProbability &origin)
void gillespieUpdateMap()
void gillespieSampleIndividuals()
void gillespieCoalescenceEvent(GillespieProbability &origin)
void gillespieDispersalEvent(GillespieProbability &origin)
void gillespieSpeciationEvent(GillespieProbability &origin)
void gillespieLocationRemainingCheck(GillespieProbability &location)
template<typename T>
double getLocalDeathRate(const T &location) const
template<typename T>
double getLocalSelfDispersalRate(const T &location) const
void clearGillespieObjects()
void setStepVariable(const necsim::GillespieProbability &origin, const unsigned long &chosen, const unsigned long &coal_chosen)
void gillespieUpdateGeneration(const unsigned long &lineage)
void updateCellCoalescenceProbability(GillespieProbability &origin, const unsigned long &n)
void updateInhabitedCellOnHeap(const Cell &pos)
void updateAllProbabilities()
void removeHeapTop()
template<typename T>
Cell convertMapLocationToCell(const T &location) const
void createEventList()

Calculates the times for each event and sorts the event list.

void sortEvents()
template<bool restoreHeap = true>
void addNewEvent(const unsigned long &x, const unsigned long &y)
void addLocation(const MapLocation &location)

Calculates the probabilities of coalescence, dispersal and speciation.

Parameters
• location: the location to add and calculate values for

void setupGillespieProbability(GillespieProbability &gp, const MapLocation &location)
void fullSetupGillespieProbability(GillespieProbability &gp, const MapLocation &location)
double calculateCoalescenceProbability(const MapLocation &location) const
template<typename T>
double convertGlobalGenerationsToLocalGenerations(const T &location, const double g) const
unsigned long selectRandomLineage(const MapLocation &location) const
pair<unsigned long, unsigned long> selectTwoRandomLineages(const MapLocation &location) const
vector<unsigned long> detectLineages(const MapLocation &location) const
void assignNonSpeciationProbability(const unsigned long chosen)

Sets the speciation probability of the provided lineage so that speciation has a random chance of occurring, but didn’t occur during this branch of the coalescence tree.

Parameters
• chosen: the index in the active lineages to set the speciation probability for

void importSimulationVariables(const string &configfile)

Import the simulation variables from the command line structure.

This function parses the simulation variables, imports them from the config file, checks that the input files exist and checks for any paused simulations. The flags are then set correctly, meaning that setup() and runSimulation() can be run immediately afterwards.

Parameters
• configfile: the path to the config file containing parameters to parse.

void importSimulationVariables(ConfigParser config)

Import the simulation variables from a ConfigOption.

This function parses the simulation variables, imports them (from either the command line or a config file), checks that the input files exist and checks for any paused simulations. The flags are then set correctly, meaning that setup() and runSim() can be run immediately afterwards.

Parameters
• config: the set of config parameters to import

void wipeSimulationVariables()

Resets all the simulation variables.

void internalSetup(shared_ptr<SimParameters> sim_parameters_in)

Sets up the simulation parameters from the one provided.

Intended for usage with metacommunity application. No output directory is expected.

Parameters
• sim_parameters_in: the simulation parameters to set up the simulation with

bool checkOutputDirectory()

Asserts that the output directory is not null and exists. If it doesn’t exist, it attempts to create it.

Return
true if output creates successfully
Exceptions
• Fatal_Exception: if the output directory creation fails

void checkSims()

Checks for existing paused simulations to resume from Sets bPaused if there are.

This version uses the default values read from the config file.

void checkSims(string output_dir, long seed, long job_type)

Checks for existing paused simulations to resume from.

Sets bPaused if there are.

This version uses the values supplied to the function directly

Parameters
• output_dir: the output directory to check for
• seed: the seed for paused sims
• job_type: the job_type for paused sims

void setProtractedVariables(double speciation_gen_min, double speciation_gen_max)

Sets the protracted variables.

Parameters
• speciation_gen_min: the minimum number of generations to have passed before speciation is allowed
• speciation_gen_max: the maximum number of generations a lineage can exist for before it is speciated.

bool hasPaused()

Gets the has_paused variable for resuming sims.

Return
if the simulation has paused

vector<double> getTemporalSampling()

Gets the map autocorrel times.

long long getSeed()

Getter for the simulation seed.

Return
Returns the seed

long long getJobType()

Gets the job type for the simulation. This is a reference number for the jobs.

Return
Returns the job type

void setSeed(long long seed_in)

Sets the simulation seed for the random number generator.

This function should only be called once.

The seed is set within the NR object. This will be fixed for the simulation and is only performed once.

Parameters
• seed_in: the desired seed to set for the simulation

double getGeneration() const

Gets the generation timer for the simulation.

Return
the number of generations that have passed

unsigned long setObjectSizes()

Sets the sizes of grid, active and data, based on the number of individuals counted from the samplemask.

Return
a count of the number of individuals that exist in the simulation

void setInitialValues()

Sets the starting values for required parameters.

void setSimStartVariables()

Sets the variables at the start of a simulation for temporary data.

This is not the main set-up routine, which creates the permanent data structures including maps, the coalescence tree and active lineage listings.

void printSetup()

Prints the statement for the setup initiation.

This is stored in a separate function so that it can be called in isolation by child classes.

void setTimes()

Sets the temporal sampling points from the time config file.

void determineSpeciationRates()

Determines the speciation rates to apply and then applies them to the coalescence tree post-simulation.

Detects speciation rates from the config files supplied.

void addSpeciationRates(vector<long double> spec_rates_in)

Adds the speciation rates to those to be applied.

void generateObjects()

Assigns the objects sizes in memory and fills with the starting lineages.

void runSingleLoop()

Runs a single loop of the simulation.

bool runSimulation()

Run the entire simulation given the start conditions already defined by setup().

Setup is assumed to have been run already. This function is the main function containing the main loop of the simulation. At the end of the simulation, returns true if the simulation is complete, false otherwise.

Return
true if the simulation has completed

bool runSimulationNoGillespie()

Runs the simulation using the standard coalescence algorithm.

Return
true if the simulation has completed

void writeSimStartToConsole()

Writes to the console that the simulation is beginning.

void writeStepToConsole()

Write the step counter to console. This function should only be called in debugging mode.

void chooseRandomLineage()

Chooses a random lineage from active.

The index of the random lineage is stored in this_step, as chosen. Also records the required variables for the step process, like x, y position.

void speciation(const unsigned long &chosen)

Speciation to supplied lineage.

Also calls the removeOldPos() and switchPositions() functions for removing the lineage out of active reference.

Parameters
• chosen:

void speciateLineage(const unsigned long &data_position)

Performs the actual speciation.

Parameters
• data_position: the position in the array of TreeNodes for this lineage

bool calcSpeciation(const long double &random_number, const long double &speciation_rate, const unsigned long &no_generations)

Calculates the speciation probability from the random number, speciation rate and number of generations a lineage has existed for.

Return
if true, speciation has occured
Parameters
• random_number: the generated random number from 0-1
• speciation_rate: the speciation rate to be applied
• no_generations: the number of generations a lineage has existed for

void coalescenceEvent(const unsigned long &chosen, unsigned long &coalchosen)

Perform the coalescence between lineages. Once coalesced, lineages are removed from the active scope.

Parameters
• chosen: the chosen lineage for coalescence
• coalchosen: the target lineage for coalscence

void checkTimeUpdate()

Checks if the number of lineages should be expanded at another sample point.

bool checkProportionAdded(const double &proportion_added)

Randomly checks if a lineage should be added, based on the proportion added.

Return
true if the lineage should be added, false otherwise
Parameters
• proportion_added: the proportion of lineages that should be added

void checkSimSize(unsigned long req_data, unsigned long req_active)

Checks the size of the main active and data objects is large enough.

Parameters
• req_data: the required data object size
• req_active: the required active object size

void makeTip(const unsigned long &tmp_active, const double &generation_in, vector<TreeNode> &data_added)

Sets the active reference to a tip, if it isn’t one already. Otherwise, creates a new tip for the new generation time.

Parameters
• tmp_active: the reference in active
• generation_in: the generation to set for the new lineage
• data_added: vector of lineages to add to data

void convertTip(unsigned long i, double generationin, vector<TreeNode> &data_added)

Creates a new reference in data containing the tip with a new generation counter.

Parameters
• i: the reference in active of the lineage to make a tip
• generationin: the generation to make the lineage a tip at
• data_added: vector of lineages to add to the data object

bool stopSimulation()

Finalises the simulation, and performs the correct tasks depending if the sim has been paused or finished.

Return

void applySpecRate(long double sr, double t)

Applies the given speciation rate to the tree.

Parameters
• sr: the required speciation rate
• t: the required time of speciation

void applySpecRate(long double sr)

Overloaded version of applySpecRates for the default generation (0.0).

Parameters
• sr: the speciation rate to apply to the tree

void applySpecRateInternal(long double sr, double t)

Applies the given speciation rate to the tree, but does not output to a file. Instead returns a pointer to the nodes object.

Parameters
• sr: the required speciation rate
• t: the required time of speciation

shared_ptr<vector<unsigned long>> getCumulativeAbundances()

Gets the sorted cumulative species abundances from the contained TreeList.

For use with metacommunity applications

Return
row of cumulative species abundances

shared_ptr<map<unsigned long, unsigned long>> getSpeciesAbundances(const unsigned long &community_reference)

Gets the species abundances from the internal tree.

Return
the species abundances
Parameters
• community_reference: the community reference

shared_ptr<vector<unsigned long>> getSpeciesAbundances()

Gets the species abundances from the internal tree.

Return
the species abundances

ProtractedSpeciationParameters setupCommunity()

Sets up Community member to point to the same output database as the simulation.

Return
the protracted speciation parameters to set up

void setupCommunityCalculation(long double sr, double t)

Sets up the generation of the tree object.

Parameters
• sr: the required speciation rate
• t: the required time of speciation

void applyMultipleRates()

Applies multiple speciation rates to the coalescence tree, ignoring repeated speciation rates.

Speciation rates are read from the speciation_rates object, which should have already been calculated.

bool getProtracted()

Gets the protractedness of the simulation. Overridden by protracted child classes.

Return

string getProtractedVariables()

Gets the protracted variables and returns them as a single, newline separated string. This method is intended to be overridden in derived classes.

Return
string containing the protracted variables, separated by newlines.

double getProtractedGenerationMin()

Gets the minimum number of generations a lineage must exist.

Without overriding this function, should always return 0.0.

Return
double the number of generations a lineage must exist

double getProtractedGenerationMax()

Gets the maximum number of generations a lineage can exist.

Without overriding this function, should always return 0.0 (no maximum).

Return
double the number of generations a lineage must exist

void sqlOutput()

Copy the in-memory database to file.

This function should not be called if the database is already opened on disc, and won’t do anything if it is.

void createAndOutputData()

Creates the output data objects and outputs important simulation data to a database file.

void outputData()

Outputs the simulation data (including the coalescence tree) to an sql database.

void sortData()

Sort and process the species list so that the useful information can be extracted from it.

void writeTimes()

Writes the times to the terminal for simulation information.

void openSQLDatabase()

Opens a connection to the in-memory database, or the on-disk database, depending on the compilation options.

void sqlCreate()

Generates the SQL database file from the full simulation data. This allows for greater analysis of the data after completion of the simulation.

void setupOutputDirectory()

Creates the output directory, if it doesn’t already exist, and deletes any existing database with the output name.

void sqlCreateSimulationParameters()

Creates the SIMULATION_PARAMETERS table in the SQL database.

string protractedVarsToString()

Outputs the protracted variables to a string.

This function is intended to be overridden by derived classes. It is intended the output is used for writing to SQL databases.

Return
string containing a list of the protracted speciation variables.

shared_ptr<ofstream> initiatePause()

Checks the output folder exists and initiates the pause.

Return
the output file stream to save objects to

void dumpMain(shared_ptr<ofstream> out)

Saves the main simulation variables to file.

Parameters
• out: the output file stream to save the object to

void dumpActive(shared_ptr<ofstream> out)

Saves the active object to file.

Parameters
• out: the output file stream to save the object to

void dumpData(shared_ptr<ofstream> out)

Saves the data object to file.

Parameters
• out: the output file stream to save the object to

void completePause(shared_ptr<ofstream> out)

Completes the pause routine and outputs the sql dump.

Parameters
• out: the output stream to close up

void setResumeParameters(string pausedir, string outdir, unsigned long seed, unsigned long job_type, unsigned long new_max_time)

Sets the resume variables so that the simulation can be resumed.

The pause directory can be the same as the output directory if it is desirable to save to the same location.

Parameters
• pausedir: the directory containing the pause folder for resuming the simulation
• outdir: the directory to write simulation output to
• seed: the simulation seed
• job_type: the simulation job reference number
• new_max_time: the maximum simulation time to run for in seconds (0 keeps old simulation max time)

void setResumeParameters()

Sets the resume variables to the defaults.

shared_ptr<ifstream> openSaveFile()
void loadMainSave(shared_ptr<ifstream> in1)

Loads the main simulation parameters from the save file into memory.

void loadDataSave(shared_ptr<ifstream> in1)

Loads the data object from the save file into memory.

void loadActiveSave(shared_ptr<ifstream> in1)

Loads the active object from the save file into memory.

void initiateResume()

Checks for resuming and prints to the terminal.

Protected Attributes

DispersalCoordinator dispersal_coordinator
shared_ptr<ActivityMap> death_map
shared_ptr<ActivityMap> reproduction_map
string fine_map_input
string coarse_map_input
string historical_fine_map_input
string historical_coarse_map_input
shared_ptr<Landscape> landscape
Matrix<SpeciesList> grid
unsigned long desired_specnum
DataMask samplegrid
double gillespie_threshold
Matrix<GillespieProbability> probabilities
vector<GillespieHeapNode> heap
Matrix<unsigned long> cellToHeapPositions
Matrix<double> self_dispersal_probabilities
unsigned long global_individuals
double summed_death_rate
shared_ptr<vector<TreeNode>> data
unsigned long enddata
shared_ptr<SimParameters> sim_parameters
shared_ptr<RNGController> NR
vector<long double> speciation_rates
bool seeded
long long seed
long long job_type
string times_file
vector<double> reference_times
bool uses_temporal_sampling
time_t start
time_t sim_start
time_t sim_end
time_t now
time_t sim_finish
time_t out_finish
time_t time_taken
vector<DataPoint> active
unsigned long endactive
unsigned long startendactive
unsigned long maxsimsize
Community community
long steps
unsigned long maxtime
double generation
double deme
double deme_sample
long double spec
string out_directory
shared_ptr<SQLiteHandler> database
bool sim_complete
bool has_imported_vars
Step this_step
string sql_output_database
bool bFullMode
bool bResume
bool bConfig
bool has_paused
bool has_imported_pause
bool bIsProtracted
string pause_sim_directory
bool using_gillespie

Protected Static Attributes

const unsigned long UNUSED = static_cast<unsigned long>(-1)