classmethod %ActivateNewShards(pNamespace As %String) as %Status
Propagate definitions of existing sharded tables to newly assigned
shards, and add the number of newly assigned shards to the visible shard count, making them
available so that any newly defined sharded entities will be sharded across the full set
of shards including the new ones. (Note: propagating table definitions to newly-assigned shards
is accomplished by calling VerifyShards() to map the new shard namespaces' routines databases to the
master routines database. Originally, before Unified Shard Databases architecture, table definitions were
copied to the shards, stored separately in each shard's routines database.)
classmethod %AssignMirroredShard(pNamespace As %String, pMirrorName As %String, pPrimaryHost As %String, pPrimaryPort As %Integer, pBackupHost As %String, pBackupPort As %Integer, pShardNamespace As %String, pMirrorDBName As %String = "", pVIP As %String = "")
Assign a mirrored shard to the sharded namespace specified by pNamespace.
The mirrored set specified by pMirrorName is expected to already exist, the the namespace specified by
pShardNamespace is expected to already exist on both failover members of that mirror, defined over
a mirrored database whose name is optionally specified by pMirrorDBName. If a VIP (Virtual IP Address) is
configured for this mirror, specify it via optional argument pVIP.
classmethod %AssignShardAppServer(pMasterNamespace As %String, pShardNumber As %Integer, pAppServerHost As %String, pAppServerPort As %Integer, pAppServerNamespace As %String) as %Status
Assign a shard application server. A shard may be assigned zero or more
application servers. The specified shard must have already been assigned,
and the specified application server instance and namespace are expected to
be configured separately from this call.
pMasterNamespace: The master namespace owning the shard to which this application
server is assigned.
pShardNumber: The number of the shard to which this application server is
assigned (this is a number between 1 and shardCount).
pAppServerHost: The hostname of the app server (may be "localhost", or an
IP address).
pAppServerPort: The default (i.e. superserver) port of the app server.
pAppServerNamespace: The namespace on the application server, whose default
database is expected to be a remote database mapped to the database of the
shard to which this application server is assigned.
classmethod %AssignShards(pNamespace As %String, pShardList As %List) as %Status
Assign a list of shards to the master namespace specified by pNamespace.
Each shard is specified by host and port of the machine / InterSystems IRIS instance that hosts the shard, and the namespace
on that instance that is the shard. Shard machines, InterSystems IRIS instances, and namespaces are assumed to be
provisioned/created separately, but do not need to exist when this method is called.
This method is expected to be called (directly or indirectly) from the provisioning/configuration interface,
which is expected to verify that the specified instances and namespaces are available to assign as shards;
that the specified namespaces are not already assigned as shards to a different logical application
server namespace; and that none of the specified shard namespaces is the namespace specified by
pNamespace (i.e. a namespace cannot be a shard of itself).
The list of shards, specified by pShardList, is in the format $lb(host1,port1,namespace1,...hostN,portN,namespaceN).
classmethod %BrokerShardOp(pStorageModel As %String, pOp As %String, pClientVersion As %Integer = 0, pMaxRanges As %Integer = 32768) as %List
Broker a sharded operation for a specified entity, of a specified storage model, in the current namespace.
Return a possibly transformed version of the operation text, suitable for sending to shards, and a list of
the shards to which it can be sent. This may be multiple shards or a single shard. When it is a single shard,
it may actually be the address of the master namespace (because the specified operation cannot be executed as
a brokered sharded operation, or cannot be sharded at all, and therefore must be executed on the master).
It is the caller's responsibility to notice if a single returned address matches the connection the caller is
already using.
When storage model is SQL:
pOp name is a SQL DML statement (SELECT, INSERT, UPDATE, or DELETE).
pClientVersion determines whether caller is new enough to support various capabilies.
0 (default) - Initial version
1 - Adds support for cosharding by foreign id
2 - Return security tokens for each shard
3 - Return username for each shard, along with security token
pMaxRanges - Maximum number of shard map ranges to return, in initial reply. Default: 32768
On success, returns a $list consisting of:
Status (1 for success)
Operation text to send to specified shards, which may be transformed or may be identical to pOp.
Server %BrokerShardOp version (always <0; If >=0, means this field doesn't exist, we're looking
at the next field, number of shard key fields). This field is only returned if pClientVersion
is >=2.
-1: The first version that adds extra output fields for pClientVersion=2.
Number of shard key fields (0 if shard key is system-assigned, or if there is no shard key)
For each key field:
Field ordinal (SQL column number, adjusted to be 0-based)
Field name
Number of shards
For each shard:
Hostname
Port
Namespace
Username (only present if pClientVersion >=2 and server %BrokerShardOp version < 0)
Security token (only present if pClientVersion >=2 and server %BrokerShardOp version < 0)
Lower bound of hash map range (This is an unused placeholder if shard key is system assigned,
or if there is no shard key)
If shard key is is foreign id:
Total number of shard map entries (i.e. number of already-assigned id ranges (>=0).
Number of shard map entries returned: min(total number of entries, pMaxRanges)
Range size.
Extent name of shard map owner, to pass as pShardMapOwner argument to %GetShardMapRanges()
Shard map entries, as a byte string. The ordinal positions of bytes in this string (starting from 1)
represent id range numbers, and the value of each entry is the shard number to which that id range maps.
range number = (id\range size)+1
NOTE: The case of shard key being foreign id is identified by the number of shard key columns being 1, and
the returned $list containing elements beyond the shard addresses.
On error, returns a $list consisting of:
Status code
Error message
Returns the number of shards to which the specified command must be sent
classmethod %DeassignShardAppServer(pMasterNamespace As %String, pShardNumber As %Integer, pAppServerHost As %String, pAppServerPort As %Integer, pAppServerNamespace As %String, pRebalancePass=0, pCleanup As %Boolean = 1) as %Status
De-assign a shard application server. An error is returned if the specified shard
has not been assigned, if the specified app server has not been assigned to this
shard, or if the specified master namespace is not a sharded namespace.
pMasterNamespace: The master namespace owning the shard to which this application
server is assigned.
pShardNumber: The number of the shard to which this application server is
assigned (this is a number between 1 and shardCount).
pAppServerHost: The hostname of the app server (may be "localhost", or an
IP address).
pAppServerPort: The default (i.e. superserver) port of the app server.
pAppServerNamespace: The namespace on the application server.
JMM1184: Added pCleanup argument
Disconnect an app-server-side Sharding Manager connection. This closes any underlying messaging connections to
shard servers.
classmethod %EndParallelLoad(pStorageModel As %String, pEntityName As %String, pSetIdSequence As %Boolean) as %Status
End parallel load for a specified entity, of a specified storage model, in the current namespace.
Set entity's idsequence to the new highest id (TBD: this is currently SQL-specific).
TBD: Release appropriate lock, or equivalent, to allow resumption of operations that are not supported during
parallel load.
Execute a command, encapsulated as a ShardableEntity, across the sharded cluster.
Sets reference parameter pInstanceCount to the number of instances (e.g. rows, objects, documents) affected.
Certain commands also set reference parameters pRowId and pSQLCODE.
classmethod %ExecLocation(pMsgCode As %String) as %String
classmethod %FactorQuery(query As %String(MAXLEN=""), maxFactors As %Integer) as %List
classmethod %GetMasterExtendedReferencePrefix()
Get a string identifying the server and directory of the master database,
that can be used as a prefix in extended global references to globals in that
database, e.g. ^|%GetMasterExtendedReferencePrefix()|myGlobalName
JMM968: With Unified Shard DBs, this is only needed to form lock names in the master
database, based on shard-local globals which, because they are mapped to the
shard database, require an explicit extended reference to refer to them in
the master database. (This is only used for centralized locking to enfore
unique constraints across shards.)
classmethod %GetShardMapRanges(pStorageModel As %String, pShardMapOwner As %String, pFirstRange As %Integer = 1, pMaxRanges As %Integer = 32768)
Get shard map ranges for a table whose shard key is the system-generated rowid.
Arguments:
pStorageModel - "SQ", for SQL tables (nothing else supported currently)
pShardMapOwner - extent name of the table owning this shard map
pFirstRange - Start returning ranges from this range number (default 1)
pMaxRanges - Maximum number of shard map ranges to return, in initial reply. Default: 32768
Returns %list whose elements are:
Status
Total number of shard map entries (i.e. number of already-assigned id ranges (>=0).
Number of shard map entries returned: min(total number of entries, pMaxRanges)
Range size.
Shard map entries, as a byte string. The ordinal positions of bytes in this string (starting from 1)
represent id range numbers, and the value of each entry is the shard number to which that id range maps.
range number = (id\range size)+1
classmethod %GetShardServerNamesAndDirectories(pShardCount As %Integer, pSkipShard As %Integer = "", ByRef pShards, pFirstShard As %Integer = 1, pGetAltDirectories As %Boolean = 0) as %Status
classmethod %InvalidateAllMappings(pNamespace As %String) as %Status
EM calls this method when it has reset configured mappings on the shasrds, letting sharding
manager know that it should reset all metadata indicating that mappings have been done, so that
they will be re-done when next needed. Call this once per application namespace that
owns shards.
classmethod %IsEntitySharded(pStorageModel As %String, pEntityName As %String) as %Boolean
Return 1 if specified entity exists, and is sharded, in the current namespace.
Else return 0.
classmethod %IsShardedEntity(pStorageModel As %String, pEntityName As %String, pVerbose=0)
Return true if the specified entity is sharded and belongs to the specified storage model.
List shard server and combining query jobs currently running on this instance.
classmethod %ListShardAddresses(pNamespace As %String = $namespace, pClientProtocolVersion As %Integer = 1, pIncludeBrokerToken As %Boolean = 1) as %List
method %ListShards(pMasterNamespace As %String = "") as %Status
List all shards configured in the current sharded cluster.
classmethod %MapUnifiedDb(pShardNumber As %Integer = 0, pQueryShardNumber=0) as %Status
On all shards, or a specified shard, map the globals and routines databases to the master's globals
and routines databases, and wildcard-map all globals of the shard-local package, and ^IRIS.SM.Shard,
to the shard's original local database, or, for query shards, to the original local database of
the corresponding data shard.
Each shard, in processing this message, will store its local directory and database name
in $$$SSGlobal($$$SSShardsSubscript,,$$$SSDirectorySubscript) and
$$$SSGlobal($$$SSShardsSubscript,,$$$SSDirectorySubscript), making it possible to
use extended global references to remotely access data in that shard (e.g. when joining multiple
sharded tables), and making it possible for query shards to map their globals database to the local
database of the corresponding data shard.
A shard's original globals and routines database names are stored in ^IRIS.SM so they can be restored
if the shard is deassigned. For data shards, these are stored in
$$$SSGlobal($$$SSShardsSubscript,,$$$SSDatabaseSubscript) and
$$$SSGlobal($$$SSShardsSubscript,,$$$SSRoutinesDBSubscript) respectively.
For query shards, these are stored in
$$$SSGlobal($$$SSShardsSubscript,$$$SSAppServersSubscript,,,$$$SSDatabaseSubscript) and
$$$SSGlobal($$$SSShardsSubscript,$$$SSAppServersSubscript,,,$$$SSRoutinesDBSubscript)
respectively.
classmethod %NextAIRange(pStorageModel As %String(MAXLEN=2), pEntityName As %String, ByRef pRangeLow As %Integer, ByRef pRangeHigh As %Integer) as %Status
Get the next range of auto_increment values for a specified entity, where a chunk is a range of a fixed size that is a
multiple of 64k. This is used to allocate ranges of auto increment values to shards of a sharded entity,
to ensure that the sequences are locally dense within 64k chunks, while
also ensuring that ids are unique across all shards, for this entity.
classmethod %NextIdRange(pStorageModel As %String(MAXLEN=2), pEntityName As %String, ByRef pRangeLow As %Integer, ByRef pRangeHigh As %Integer) as %Status
Get the next range of ids for a specified entity, where a chunk is a range of a fixed size that is a
multiple of 64k. This is used to allocate ranges of system-assigned ids to shards of a sharded entity,
to ensure that the sequences are locally dense within 64k chunks, for bitmap index efficiency, while
also ensuring that ids are unique across all shards, for this entity.
If the id is the shard key for this entity, add a node to this entity's shard map, to map this range
to the shard which allocated it.
Inherited description: This callback method is invoked by the %New() method to
provide notification that a new instance of an object is being created.
If this method returns an error then the object will not be created.
It is passed the arguments provided in the %New call.
When customizing this method, override the arguments with whatever variables and types you expect to receive from %New().
For example, if you're going to call %New, passing 2 arguments, %OnNew's signature could be:
Method %OnNew(dob as %Date = "", name as %Name = "") as %Status
If instead of returning a %Status code this returns an oref and this oref is a subclass of the current
class then this oref will be the one returned to the caller of %New method.
Purge cached information about a specified shard.
Call this if a shard instance is deleted, and then recreated with the same
host, port, and namespace, to be assigned to the same shard number. This
ensure that we correctly treat the recreated instance as a new, uninitialized
shard.
classmethod %ReassignShard(pMasterNamespace As %String, pShardHost As %String, pShardPort As %Integer, pShardNamespace As %String, pShardNumber As %Integer, pShardMirrorName As %String, pShardBackupHost, pShardBackupPort, pShardVIP As %String) as %Status
%ReassignShard should only be called from the %SYSTEM.Sharding ReassignShard method
classmethod %RemoveShards(pNamespace As %String, pShardList As %List = "", pRebalancePass=0, pCleanup As %Boolean = 1) as %Status
Remove one or more shards from the set of shards configured for a specified namespace.
The shards to be removed are specified as a list in the format $lb(hostname, port, namespace [,...]).
An error is returned if any sharded entities have been defined in these shards. This method does
not delete the shard namespaces themselves, nor any data in them. If there are any shard
app servers (aka query shards) assigned to the shard being removed, they are deassigned.
THIS IS NOT SAFE TO USE IN PRODUCTION. This is provided only as a convenience during initial
testing and development. 1/25/17: This may now be safely used via $SYSTEM.Sharding.DeassignShard,
which performs appropriate checks.
JMM1109: pRebalancePass controls behavior as follows:
0: Not called from Rebalance(). If any sharded tables exist, just flag this shard for pending removal.
If no sharded tables exist, remove the shard's metadata, close the gap in shard number, and cleanup the shard.
1: First pass from rebalancing: remove this shard's metadata from ^IRIS.SM("shards"), closing the gap in
shard numbering, but don't send a message to cleanup the shard (so mappings are still available for any
already-running queries). This is called while queries are using ^IRIS.SM("savedshards"), so they aren't
affected by this and can still access the shards that are pending removal.
2: Second pass from Rebalance(). Use the ^IRIS.SM("savedshards") array, while still has metadata for the
shards that are pending removal. Complete the process of removing the shards, by sending messages to
clean them up (removing mappings, ^IRIS.SM.Shard, etc.). This is called when queries have switched to
using the modified ^IRIS.SM("shards") array that no longer contains entries for the removed shards.
JMM1184: Added pCleanup argument
classmethod %TrueOnAllShards(pCommand As %String, ByRef pResult, pSkipCurrentShard As %Boolean = 0) as %Status
Execute on all shards an object script command which sets the variable SNOutput
to a non-zero or zero (or "") value. If the command sets SNOutput to a non-zero
value on all shards, set pResult to a non-zero value. If it sets SNOutput to a
zero or "" value on any shard, set pResult to a zero or "" value. If any shard
returns an error, set pResult to 0.
classmethod %VerifyShards(pMasterNamespace="", pShardNumber As %Integer = "", pQueryShardNumber As %Integer = "", pReturnFirstError As %Boolean = 0, pVerifyDataShardFirst As %Boolean = 1, pRefreshSqmMetadata=1, pRecurse=0, pSkipSetMasterBackup=0) as %Status
Verify accessibility and correct configuration of all or specified shards
pMasterNamespace - Namespace in which to validate shards, defaults to current namespace
pShardNumber - Number of shard to validate. If not specified, validate all shards.
pQueryShardNumber - If specified, validate the specified query shard of the shard
specified by pShardNumber. pShardNumber must be specified if pQueryShardNumber is specified.
pReturnFirstError: If true, return status after first shard fails verification. Else (the default)
attempt to verify all data shards, return nested status if any fail, else attempt to verify all
query shards, return nested status if any fail.
pVerifyDataShardFirst: If true, before verifying a query shard verify its data shard. This is true
when verifying individual shards, false when verifying all shards (because in the latter case,
all data shards are verified, recursively, before any query shards).
classmethod AlreadyAssigned(shard As %List, ignoreShard="") as %Boolean
classmethod DeassignQueryShard(MasterNamespace As %String = $namespace, ShardHost As %String, ShardPort As %Integer, ShardNamespace As %String, Cleanup As %Boolean = 1) as %Status
JMM1184 - Internal-only classmethod for deassigning a query shard, offering option of not
cleaning up mappings, for use when compute node being deassigned has already been destroyed
classmethod GetShardingRole(namespace As %String, ByRef dataMaster As %String, ByRef shardNumber As %Integer) as %String
Given a namespace name, if this namespace plays a role in a sharded cluster, return a string
indicating the role, as follows:
"DM" - Data Master
"AM" - Application Master
"DS" - Data Shard
"QS" - Query Shard
"" - Not part of a sharded cluster
If the namespace is part of a sharded cluster, the address of the Data Master of the cluster is returned by reference, in the form "::". If the namespace is a Data Shard or Query Shard, the shard number is returned by reference.
final classmethod ShardNumber(pId As %Integer = 1) as %Integer [ SQLProc = ShardingManager_ShardNumber ]
Projected as the stored procedure: ShardingManager_ShardNumber
final classmethod ShardNumberNS(namespace As %String = "") as %Integer [ SQLProc = ShardingManager_ShardNumberNS ]
Projected as the stored procedure: ShardingManager_ShardNumberNS
Given a namespace name, if this is a Shard namespace, return the ShardNumber
If not a shard namespace, return 0