• You can mix stream data types in one expression (short, int, long, float, double, string, boolean, ...)
• Standard operators:
  • Power: ^
  • Boolean Not: !
  • Unary Plus, Unary Minus: +x, -x
  • Dot product, cross product: ., ^^
  • Modulus: %
  • Division: /
  • Multiplication: *
  • Addition, Subtraction: +, -
  • Less or Equal, More or Equal: <=, >=
  • Less Than, Greater Than: <,>
  • Not Equal, Equal: !=, ==
  • Boolean And: &&
  • Boolean Or: ||
  • Assignment: =
  • These operators work with strings: +,<=,>=, <,>,!=,==,=
    
• Bitwise operators:
  • And: a & b
  • Or: a | b
  • Complement: ~a
  • Xor: a ** b
  • Left shift: a >> b
  • Right shift: a << b
  • Right shift with zero extension: a <<< b
• Support for functions (only numbers as arguments):
  • Trig functions: 
    • Sine: sin(x)
    • Cosine: cos(x)
    • Tangent: tan(x)
    • Arc Sine: asin(x)
    • Arc Cosine: acos(x)
    • Arc Tangent: atan(x),
    • Arc Tan with 2 params: atan2(y,x)
    • Secant: sec(x)
    • Cosecant: cosec(x)
    • Co-tangent: cot(x)
    • Hyperbolic Sine: sinh(x)
    • Hyperbolic Cosine: cosh(x)
    • Hyperbolic Tangent: tanh(x)
    • Inverse Hyperbolic Sine: asinh(x)
    • Inverse hyperbolic Cosine: acosh(x)
    • Inverse Hyperbolic Tangent: atanh(x)
  • Log and exponential functions:
    • Natural Logarithm: ln(x)
    • Logarithm base 10: log(x)
    • Logarithm base 2: lg(x)
    • Exponential (e^x): exp(x)
    • Power: pow(x)
  • Statistical functions:
    • Average: avg(x1, x2, x3, ...)
    • Minimum: min(x1, x2, x3, ...)
    • Maximum: max(x1, x2, x3, ...)
    • Vector Sum: vsum(x1, x2, x3, ...)
  • Rounding functions:
    • Round: round(x), round(x, p)
    • Round to integer: rint(x), rint(x, p)
    • Floor: floor(x)
    • Ceiling: ceil(x)
  • Other functions
    • if: if(cond, trueval, falseval)
    • Convert number to string: str(x)
    • Absolute Value /Magnitude: abs(x)
    • Random number (between 0 and 1): rand()
    • Modulus: mod(x,y) or x%y
    • Square Root: sqrt(x)
    • Sum: sum(x1, x2, ...)
    • Binomial coefficients: binom(n,i)
    • Signum (-1, 0, 1 depending on sign of argument): signum(x)
    • Is Null: isNull(x)
  • String functions:
    • Left most characters: left(str, len)
    • Right most characters: right(str, len)
    • Middle (zero based): mid(str, start, len)
    • Substring (zero based): substr(str, start, [end])
    • Lower Case: lower(str)
    • Upper Case: upper(str)
    • Length: len(str)
    • Trim: trim(str)
    • Replace All: replaceall(str, regex, replacement)
    • Replace First: replacefirst(str, regex, replacement)
    • Convert String to a Number in a Given Base: fromBase(str, base)
  • JSON functions: isJSON(str, optional: "VALUE | SCALAR | ARRAY | OBJECT") Determine whether a given string is valid JSON.
    
    Specifying the optional type argument puts a constraint on which type of JSON object is allowed. If the string is valid JSON, but not that type, false is returned. The default is "VALUE". true = isJSON(1)
    true = isJSON("1")
    
    • is JSON: isJSON(str, "VALUE | SCALAR | ARRAY |OBJECT")
  • Time:
    • Current Time in epoch milliseconds: time()
    • Formats epoch milliseconds date time into a string: dtFormatter(epochTime, pattern, timeZoneId)
  • Constants:
    • pi
    • e
    • NULL: Return NULL to indicate that the interval is a Gap.
  • System Variables:
    • LAST_VALUE: The derived stream's last value prior to the current calculation. Tthe tail of the derived stream.
    • SAMPLE_TIME: The time of the regular stream sample being calculated in epoch Milliseconds.
    • SAMPLE_SDTIME: The start date time of the interval stream interval being calculated in epoch Milliseconds.
    • SAMPLE_EDTIME: The end date time of the interval stream interval being calculated in epoch Milliseconds.

• 1
  • This is a derived stream with no variables. All interval values are set to one during derivation. This kind of derivation is convenient for testing.
• 2*(n+3)
• n^2
• left("test string", 4)
• 2-cos(n)
• if(n<n2, true, false)
• if (n > n2, "stream1 value has exceeded stream2 value", "stream1 value is equal to or less than stream2 value")
• dtFormatter(SAMPLE_TIME, "MM/dd/YYY", "US/Central")

When does derivation occur?
GroveStream's will periodically detect (usually every few seconds) when all dependent stream intervals have arrived before doing a derivation. By default, all dependents must have data arrive since the last derivation for derivation to occur. Derivation will also occur if the advanced setting Perform derivation when is set to Any dependent values have arrived and at least one dependent value arrives since the last derivation. Derivation of an interval may be performed several times depending on the size of the interval and the size of its dependent intervals. If new (not updated) dependent intervals arrive and fall within the interval time range being derived, then the derived interval will be calculated with the new interval values. The algorithm for this is a bit complicated as an interval might be derived from another derived variable which is derived from smaller intervals. GroveStreams tracks the smallest dependent interval size end datetime and uses that value within the algorithm to determine interval derivation.

Example: A derived stream has one day cycles but its dependents are one hour cycles. The dependent one hour cycles arrive at different times. The derivation engine will update the derived one day interval several times during a day - whenever all dependent one hour intervals exist for an hour that hasn't been used in a prior derivation. 

A stream has three internal datetimes: start, end and completed. Start is the start datetime of the first interval in the store. End is the end datetime of the last interval in the store. Completed datetimes are used for derivation and are the earliest completed datetime of all dependent streams. If a stream is not a derived stream then its end datetime will be equal to its completed datetime.

Derivation occurs for a stream when all of its dependent streams have completed datetimes that are less than the derived streams' completed datetime (unless Any dependent values have arrived is selected then only one needs to be less). Once a derived interval's end datetime is less than or equal to the streams' completed datetime the derived interval enters a "closed" state. It will not be derived again and derivation moves on to the next set of intervals in the cycle. 

Although a derived interval in a "closed" state will not be re-derived but it can be changed via the API or from within the GroveStreams user interface just like any other stream. To force a derivation on "closed" intervals, delete all interval data from a derived stream and the derivation engine will recalculate it all. Not auto deriving "closed" intervals avoids doing dependency tree look-ups every time an interval value changes (as feeds are uploaded). GroveStreams is all about performance and we feel this limitation is acceptable to keep performance high.

Cycles and Functions
A stream can be derived from other streams with different cycle sizes. If the dependent stream's cycle is smaller it will be rolled up to match the size of the derivation stream. The smaller cycle must be able to rollup evenly into the derived interval (1 second into 10 second is an even rollup - 7 second into 1 minute is uneven). The cycle function will be used for the rollup method. If the dependent stream's cycle is larger then the dependent interval with an end datetime that falls within the derived interval will be used.

You can optionally leave the cycle blank (or set to automatic in the user interface). When it is left blank, GroveStreams will select a cycle for you and perform an on-the-fly rollup if needed.

Cycles and Functions are ignored for point stream dependents.

Expressions
Expressions are composed of variables which are stream. If a variable is an interval stream then the user can choose the interval offset for that variable. For example, if an hourly interval with a time span of 2:00 pm to 3:00 pm is being calculated a dependent variables' offset can be set to -1 and that variables' data for time span 1:00 pm to 2:00 will be used in the calculation. Offsets are useful for things like rolling averages.

Example of a Derived Stream Expression that calculates one second 3 pt rolling averages from Component1.Stream1.

Variables: