XML Functions representation

59 posts / 0 new
Last post
rparrotto
Offline
Joined: 2009-10-01
XML Functions representation

I can propose a first extension of the schema in order to support user-defined functions.

The solution provides two modules, one for the functions and one for the algorithm.

FUNCTIONS:
- Function can be embedded in the XML document or external defined;
- EXTERNAL functions have attributes "name" and "language" (C, Modelica)
- EMBEDDED functions are designed with name (attribute), list of output, input, protected variables (with "type" and "default value" attributes) and algorithm; every of these sections is optional;
- it is optionally possible to define a set inverse and derivative of a functions as additional elements of "Function";
- INVERSE functions should be allowed only with one output variable and at least one input variable is necessary. The output of the inverse function should be an input of the original function and the output of the original function should be input variable of the inverse;
- DERIVATIVES of a function must have at least one output and one input variable. Input variables have the additional attributes "derived" ,"zeroDerivative", both defined as boolean, and "order" defined as integer. If derived=true then the function is derived with respect of that variable; the function is differentiated with respect of the variables in the same order they are defined (from the first to the last); if a variable has derived=true, it should also be of Real type. The derivative function is only valid if variables with zeroDerivative=true are independent of the variables the function is differentiated with respect to (i.e. the derivative of that input variable is "zero"); the attribute "order" defines the order of the derivative with respect of that variable and the default value is 1;
- an element FunctionsList is defined as an unbounded list of the previous discussed functions (embedded and external). This element will be referred by the main schema.

ALGORITHMS:
the following statements are supported:
- ASSIGN, identifier=expression;
- BREAK, that shall be used in while and for loops only;
- RETURN;
- IF, with optional (0-unbounded) elseif and (0-1) else branches. The else branch is the one taken in account if all the other conditions are not satisfied. Each branch is defined by a condition expression and a list of statements, except Else that has not condition;
- WHILE, defined by condition expression and list of statements;
- FOR loop, defined by an index and the list of statements. The index is an identifier and has the attributes StartValue, FinalValue (required) and Step (optional, 1 by default);
- FUNCTION CALL statement with the name of the called function and the list of arguments (expressions)
- ASSERTION, with a required condition (expression) and an optional message. The assertion has an optional attribute "level" that can have value "error" (default) or "warning". If level=error then the algorithm should not be executed, if level=warning the warning message is returned but the algorithm should not be skipped

Vectors and records aren't supported,at least for this first draft.

ADDITIONAL FUNCTION CALL EXPRESSION
- an expression FunctionCall has been added to the expression definition. A function call should return one value only, in order to be used in the expression. The function call is defined by the name of the function (attribute) and a list of arguments (expression type)

ADDITIONAL EQUATIONS
The list of dynamic equations has been updated, adding the element FunctionSetEquations. This element contains the equations due to the functions and it's defined by a list of output variables and a function call named as "SourceFunction". The name "set" is to remind that the element defines a set of scalar equations, one for each output variable.

In attach the updated schema modules for expression, equations, algorithms, functions, plus some graphical representations of the key elements.

Best
Roberto

jakesson
Offline
Joined: 2009-03-14
Hi, I have read your schemata

Hi,
I have read your schemata and over all I think that this is a good start, but there are some issues that needs to be addressed.
- A major complication that is not addressed in the proposal is functions with vector arguments. Since algorithms cannot be scalarized as equations, array accesses must be supported in function argument expressions and in algorithms. This has far reaching consequences. For example, slicing of arrays (e.g x[3,:].y[3,1,1:2].z[3:1,1:end]) is one complication. This means that most likely we have to introduce array subscripts for identifiers. Very similar problems arise in the C code generation, once we get to the point where we design code generation for functions we probably can get some more input to the XML schema design.
- Variable declarations in functions have names (needed to sort out which named argument binds to which function variable) and may also have binding expressions, even output variables. Variable declarations may also have attributes.
- The definition of Index in For statements does not support statements like
for i in {3,1,2} loop
...
end loop;
- I am not sure about how the definition of FunctionSetCall works (I assume that this construct corresponds to calls like (x,y)=f(v,u)), perhaps it will become clearer if you provide a screen dump. Also, I think that the name FunctionCallEquation would be a more descriptive name. Also, a counterpart for algorithms is needed, FunctionCallStatement.
 - Also functions with several output arguments may be called from within expressions - only the first argument is then used.
Best
/Johan

rparrotto
Offline
Joined: 2009-10-01
binding exp as function argument
jakesson wrote:

Variable declarations in functions have names (needed to sort out which named argument binds to which function variable) and may also have binding expressions, even output variables

I added an optional attribute "argumentValue" to "Identifier" that should be used when the identifier is used for a binding expression in a function call´s argument... ok?

jakesson
Offline
Joined: 2009-03-14
XML for function feedback
rparrotto wrote:
jakesson wrote:

A major complication that is not addressed in the proposal is functions with vector arguments. Since algorithms cannot be scalarized as equations

I agree and we should discuss this point. During the Skype meeting of December we decided to don't handle vector arguments at least for the first results.

I recall this, we'll postpone the design decisions for this.

rparrotto wrote:
jakesson wrote:

The definition of Index in For statements does not support statements like for i in {3,1,2} loop ... end loop;

I think we should avoid it. If I have good memory I talked about this also in the famous meeting. Since we want a general representation also for algorithm, I think that we should choose one and only one representation for every kind of statement, the most general as possible, that I think it is the one I've chosen. Not all languages accept that kind of For statement and we don't want a representation of Modelica algorithms, but of general algorithms. Then I think that the handling of this kind of Modelica statement is a problem of mapping that should be solved by the XML code generator of the Modelica tool.

I don't immediately see that the proposed form index_start:step:index_end is the most general form - how do you map something like {1,-4,34,4} onto a range expression? I agree that this is cumbersome but that is Modelica :). In the JModelica grammar the for indices are given by FExps in the most general case. I would imagine that in the C code generation we will have to introduce something like iteration sets.

rparrotto wrote:
jakesson wrote:

Also functions with several output arguments may be called from within expressions - only the first argument is then used.

Yes, I know that it's possible in Modelica, but I hate this thing! It's so inelegant :D I want also other feedbacks on this.

Inelegant or not, it's a reality in Modelica. Which means we have to support it :).
Best
 
/Johan

rparrotto
Offline
Joined: 2009-10-01
Reply on Johan's comments
jakesson wrote:

I am not sure about how the definition of FunctionSetCall works (I assume that this construct corresponds to calls like (x,y)=f(v,u)), perhaps it will become clearer if you provide a screen dump

I suppose you are talking about FunctionEquationsSet (correct me if I'm wrong, but I can't find FunctionSetCall). The screenshot is in attach. The goal is to represent the dynamic equations due to user-defined functions like (x,y)=f(v,u) as you guessed. It should be interpret like a new equation for every ScalarOutput element in the document. If you have better ideas, you're welcome.

jakesson wrote:

I don't immediately see that the proposed form index_start:step:index_end is the most general form - how do you map something like {1,-4,34,4} onto a range expression? I agree that this is cumbersome but that is Modelica :). In the JModelica grammar the for indices are given by FExps in the most general case. I would imagine that in the C code generation we will have to introduce something like iteration sets.

According to the theorem of Bohm-Jacopini, every algorithm can be represented using only one sequence,choice and loop structures.

Assuming of having vectors, using my solution, a possible translation could be:

x[0]=1; x[1]=-4; x[2]=34; x[3]=4;
for i=0 to 3
......... do something with x[i] ..........
end for

jakesson wrote:

Also functions with several output arguments may be called from within expressions - only the first argument is then used.

rparrotto wrote:

Yes, I know that it's possible in Modelica, but I hate this thing! It's so inelegant :D I want also other feedbacks on this.

jakesson wrote:

Inelegant or not, it's a reality in Modelica. Which means we have to support it :).

Sorry, I should write a better documentation. I think we don't need to modify the schema: the signature of the function shows the list of the outputs and inputs. When we use FunctionCall in an expression, it doesn't ask any information about output variables, because it assumes that the function returns only one value.This value could be either the only value returned by the function OR the first value if the function returns several outputs.

QUESTION: I've embedded the derivative/inverse functions in the "root" function definition. Another solution would be to define the derivatives externally and use an attribute as an "annotation" to recall that it is a derivative/inverse function of another one, previously defined. What do you prefer?

AttachmentSize
functionEquationSet.png 7,94 KB
rparrotto
Offline
Joined: 2009-10-01
binding exp as function argument
jakesson wrote:

Variable declarations in functions have names (needed to sort out which named argument binds to which function variable) and may also have binding expressions, even output variables

I added an optional attribute "argumentValue" to "Identifier" that should be used when the identifier is used for a binding expression in a function call´s argument... ok?

jakesson
Offline
Joined: 2009-03-14
Default function arguments
rparrotto wrote:
jakesson wrote:

Variable declarations in functions have names (needed to sort out which named argument binds to which function variable) and may also have binding expressions, even output variables

My comment was about function arguments with default expressions, e.g.:

function foo
Real input x = 1;
Real output y = 4 + z;
protected
parameter Real z = 4;
algorithm
...
end foo;

In this case, the binding expressions need to be encoded in association with each InputVariable/OutputVariable element. The type of the binding expression should be Exp.

rparrotto wrote:

I added an optional attribute "argumentValue" to "Identifier" that should be used when the identifier is used for a binding expression in a function call´s argument... ok?

I'm not quite sure what is meant by this. It is true that identifiers in function calls referring to input function arguments can be used, i.e., named arguments may be used as in f(x,y,v=5,p=n*4). Is this what you mean?

Best
/Johan

jakesson
Offline
Joined: 2009-03-14
Functions in XML schema
rparrotto wrote:
jakesson wrote:

I am not sure about how the definition of FunctionSetCall works (I assume that this construct corresponds to calls like (x,y)=f(v,u)), perhaps it will become clearer if you provide a screen dump

I suppose you are talking about FunctionEquationsSet (correct me if I'm wrong, but I can't find FunctionSetCall). The screenshot is in attach. The goal is to represent the dynamic equations due to user-defined functions like (x,y)=f(v,u) as you guessed. It should be interpret like a new equation for every ScalarOutput element in the document. If you have better ideas, you're welcome.

I think that the design is ok, but I would prefer the name of the element to be changed to FunctionCallEquation.

rparrotto wrote:
jakesson wrote:

I don't immediately see that the proposed form index_start:step:index_end is the most general form - how do you map something like {1,-4,34,4} onto a range expression? I agree that this is cumbersome but that is Modelica :). In the JModelica grammar the for indices are given by FExps in the most general case. I would imagine that in the C code generation we will have to introduce something like iteration sets.

According to the theorem of Bohm-Jacopini, every algorithm can be represented using only one sequence,choice and loop structures. Assuming of having vectors, using my solution, a possible translation could be:

x[0]=1; x[1]=-4; x[2]=34; x[3]=4;

for i=0 to 3
.........
do something with x[i]
..........
end for

Yes, this translation is indeed possible, but I still think that it is too restricted. The "iteration sets" introduced in the translation may be very large and cumbersome to store explicitly. Now, all for loops in equation sections are eliminated (unrolled) in the scalarization step. Therefore, for loops are only of relevance in algorithms in the context of the XML schema. Since we have not yet started the design of functions with vector arguments, we may postpone also the design of for statements in algorithms for time being.

rparrotto wrote:
jakesson wrote:

Also functions with several output arguments may be called from within expressions - only the first argument is then used.

rparrotto wrote:

Yes, I know that it's possible in Modelica, but I hate this thing! It's so inelegant :D I want also other feedbacks on this.

jakesson wrote:

Inelegant or not, it's a reality in Modelica. Which means we have to support it :).

Sorry, I should write a better documentation. I think we don't need to modify the schema: the signature of the function shows the list of the outputs and inputs. When we use FunctionCall in an expression, it doesn't ask any information about output variables, because it assumes that the function returns only one value.This value could be either the only value returned by the function OR the first value if the function returns several outputs.

I agree, I don't think that we need to change the schema.

rparrotto wrote:

QUESTION: I've embedded the derivative/inverse functions in the "root" function definition. Another solution would be to define the derivatives externally and use an attribute as an "annotation" to recall that it is a derivative/inverse function of another one, previously defined. What do you prefer?

I have no strong opinion on this - Francesco?

Best

/Johan

rparrotto
Offline
Joined: 2009-10-01
binding exp

Yes, binding for input and outputs has been already improved and what I was talking about there is the arguments of function calls. Actually the solution of having just an attribute to bind a value to the identifier is not enough, because attributes can belong to simple types only (and we need also expressions). I will fix this.

jmattsson
Offline
Joined: 2009-10-18
Must have a subject
rparrotto wrote:

x[0]=1; x[1]=-4; x[2]=34; x[3]=4;
for i=0 to 3
......... do something with x[i] ..........
end for

That is how it will be solved in the C code as well, I think.

As for slices, they can be flattened out into FArrays and will then be easy to handle.

rparrotto
Offline
Joined: 2009-10-01
A more general For loop

I modified the For loop definition. Please take a look to the image in attach.

The index is now composed by an attribute "IteratingVariable" that should hold the name of the variable in the index, while an optional "IterationSet" element of type Exp has been added to represent the set where the variable should iterate. Is it ok like this? Suggestions? Is the IterationSet definition a bit ambiguous?

[img_assist|nid=217|title=ForLoop with IterationSet|desc=|link=none|align=left|width=640|height=255]

jakesson
Offline
Joined: 2009-03-14
I think that this is ok. Of

I think that this is ok. Of course, modeling the iteration index as a variable with a general FExp associated is perhaps a bit general since this includes also non valid expressions in this context. On the other hand, the receiver of XML document needs to check the type of the expression anyway.
 
Best
/Johan

rparrotto
Offline
Joined: 2009-10-01
I suggest to keep this for

I suggest to keep this for now (it works + it's easy) and then I will try to design a better solution for the iteration variable later.

jakesson
Offline
Joined: 2009-03-14
Sounds good.

Sounds good.

rparrotto
Offline
Joined: 2009-10-01
code generation

Some issues about the XML code generation of Functions.

The model I'm simulating is the following:

-------------------------------------------------------------------
package myPack
model myModel

function f
input Real x[2];
output Real y[2]={28,91};
algorithm
y:=x;
end f;

Real z[2]={1,1};
Real w[2];

equation
w=f(z);

end myModel;

end myPack;
------------------------------------------------------------------

1) The FUNCTION NAME is represented like this
< fun:Function name="myPack.myModel.f">

Would it be better to have just "f" or is it ok like this?

2) About the VARIABLES TYPE. The code generated for the input variable x is

< fun:InputVariable name="x" type="Real[2]" xsi:nil="true"/>

Is it ok to represent the type in this way? Or it would better to have type="Real" and another attribute that says it is an array? I'm thinking of which one is the best solution to have an easier import of the model.

3) VARIABLES INITIAL VALUES. The code generated for "output Real y[2]={28,91};" is just:

< fun:OutputVariable name="y" type="Real[2]" xsi:nil="true"/>

where nil="true" means that there is no initial value. It seems like there is no initial value represented in the AST, but the binding of the variable and initial values (y[1]:=28, y[2]:=91) are made at the beginning of the algorithm section as assignment nodes. It is perhaps logically correct, but I'm guessing if it is the result you wanted while designing the AST.

4) FLATTENED VARIABLES. Representing the assignment y:=x in the algorithm, the result I have is

< fun:Assign>
< exp:Identifier>y< /exp:Identifier>
< fun:Expression>
< exp:Identifier>x< /exp:Identifier>
< /fun:Expression>
< /fun:Assign>
< fun:Assign>
< exp:Identifier>y< /exp:Identifier>
< fun:Expression>
< exp:Identifier>x< /exp:Identifier>
< /fun:Expression>
< /fun:Assign>

So the reference to the index of the array generating the variable is missing. I used FIdUse.name() to get the name of the variable (and I have the correct result when they are used in equations). What's wrong?

rparrotto
Offline
Joined: 2009-10-01
If Statement

I'm generating code for the algorithm

if (x[1]==y[1]) then
x:=y;
end if;

I figured out that an else branch is always produced. Taking a look at the grammar, in fact, I noticed

FIfStmt : FIfWhenStmt ::= ElseStmt:FStatement*;

Why? ElseStmt should be completely optional, shouldn't it? (Modelica specification, par.11.2.6)

jakesson
Offline
Joined: 2009-03-14
ElseStmt
rparrotto wrote:

I'm generating code for the algorithm if (x[1]==y[1]) then x:=y; end if; I figured out that an else branch is always produced. Taking a look at the grammar, in fact, I noticed FIfStmt : FIfWhenStmt ::= ElseStmt:FStatement*; Why? ElseStmt should be completely optional, shouldn't it? (Modelica specification, par.11.2.6)

 
ElseStmt is optional, as you correctly point out. The grammar 
 
FIfStmt : FIfWhenStmt ::= ElseStmt:FStatement*;
 
means that an FIfStmt node may have zero or more ElseStmts, i.e., it is optional.
Best
/Johan

rparrotto
Offline
Joined: 2009-10-01
else branch

Yes, the problem was that an else node was always generated, but I've found the error, that it was in code generation, not in the grammar. Sorry!

jakesson
Offline
Joined: 2009-03-14
Here are my comments to your

Here are my comments to your questions:
1) I would prefer using the qualified name, rather than the short one in order to avoid name conflicts.
 
2) I suggest you represent the variable type with one attribute type="Real" and then the array subscripts as a list (containing 0 or more entries) of expressions. This separation will probably make it easier to handle the functions when doing code generation. On the other hand, this is the same issue as how to represent array indices in identifiers. It may be worthwhile considering representing identifiers using an equivalent of the FQName node.
 
3) Jesper will have to answer the question weather binding expressions are always transformed into assignments. If so, then I see no reason to generate code for binding expressions in the variable declaration nodes. Also, what is the intention of adding the attribute nil?
 
4) Using FIdUse.name() should be fine, Jesper - what is the issue here?
 
Best
/Johan

rparrotto
Offline
Joined: 2009-10-01
else branch
jakesson wrote:

FIfStmt : FIfWhenStmt ::= ElseStmt:FStatement*;
 
means that an FIfStmt node may have zero or more ElseStmts, i.e., it is optional.
Best
/Johan

mmmh, thinking about this, I'm still not sure. There is difference between having a list of statements with no statements and not having a list. It is the same difference between

if(expression) then
....list of statements....
else
.... nothing ....
end if;

and

if(expression) then
.... list of statements ...
end if;

in fact in the code generation the condition I have to solve is

if (this.getNumElseStmt()!=0){
... generate code for else statement
}

so I can't generate correct code for the first case. But it's just details I think

jakesson
Offline
Joined: 2009-03-14
ElseStmt
rparrotto wrote:
jakesson wrote:

FIfStmt : FIfWhenStmt ::= ElseStmt:FStatement*;
 
means that an FIfStmt node may have zero or more ElseStmts, i.e., it is optional.
Best
/Johan

mmmh, thinking about this, I'm still not sure. There is difference between having a list of statements with no statements and not having a list. It is the same difference between
 
if(expression) then
....list of statements....
else
.... nothing ....
end if;
and

if(expression) then
.... list of statements ...
end if;

in fact in the code generation the condition I have to solve is
if (this.getNumElseStmt()!=0){
... generate code for else statement
}
so I can't generate correct code for the first case. But it's just details I think

 
I don't think that we have to differentiate between these two cases: if the list ElseStmt is empty, then we may assume that we have the second case, even though it may have originated from code in case one.
 
Best
/Johan
 

rparrotto
Offline
Joined: 2009-10-01
variables
jakesson wrote:

3) Jesper will have to answer the question weather binding expressions are always transformed into assignments. If so, then I see no reason to generate code for binding expressions in the variable declaration nodes. Also, what is the intention of adding the attribute nil?
 

The element OutputVariable (such as Input and Protected) are defined by an expression and attributes name and type, like this:

< fun:OutputVariable name="name" type="Real" >
< exp:RealLiteral>1.2< /exp:RealLiteral>
< /fun:OutputVariable>

if the expression doesn't exist, then the element has nil value (notice that this attribute is not defined in the schema, but it is proper of XML schema language), represented by that attribute. I've used it to have a more compact code, because this allows to have the short tag representation (one line, as < element /> instead of < element>< /element>:

< fun:OutputVariable name="name" type="Real" nil="true"/>

The alternative is to explicitly create an optional element "BindingExpression" or "InitialExpression" and always have

< fun:OutputVariable name="name" type="Real" >
< fun:BindingExp>
< exp:RealLiteral>1.2< /exp:RealLiteral>
< /fun:BindingExp>
< /fun:OutputVariable>

or, if there is no expression:

< fun:OutputVariable name="name" type="Real" >< /fun:OutputVariable>

It is a matter of design: we can choose the solution we prefer: just tell me what you think...

rparrotto
Offline
Joined: 2009-10-01
variables' name
jakesson wrote:

2) I suggest you represent the variable type with one attribute type="Real" and then the array subscripts as a list (containing 0 or more entries) of expressions. This separation will probably make it easier to handle the functions when doing code generation. On the other hand, this is the same issue as how to represent array indices in identifiers. It may be worthwhile considering representing identifiers using an equivalent of the FQName node.

So, I've got your idea, a solution for "input Real x[2];" could be:

< fun:InputVariable name="x[2]" type="Real" > .....

or do you intend something else?

rparrotto
Offline
Joined: 2009-10-01
For loop index

currently the INDEX of the for loop is designed like this:

< fun:Index IterationVariable = "i" >
< fun:IterationSet>
< exp: an expression >....
< /fun:IterationSet>
< /fun:Index>

for example the code
-------------------------
for i in {1,5} loop
end loop;
------------------------

produces

< fun:For>
< fun:Index IterationVariable="i">
< fun:IterationSet>
< exp:Array>
< exp:IntegerLiteral>1< /exp:IntegerLiteral>
< exp:IntegerLiteral>5< /exp:IntegerLiteral>
< /exp:Array>
< /fun:IterationSet>
< /fun:Index>
< fun:Statements>....

But how loops like

for i in 1:4 loop
end loop;

are represented in the AST ? I generate the code from:

FForIndex : CommonForIndex ::= FVariable;
abstract CommonForIndex ::= [FExp];

jakesson
Offline
Joined: 2009-03-14
FQName
rparrotto wrote:
jakesson wrote:

2) I suggest you represent the variable type with one attribute type="Real" and then the array subscripts as a list (containing 0 or more entries) of expressions. This separation will probably make it easier to handle the functions when doing code generation. On the other hand, this is the same issue as how to represent array indices in identifiers. It may be worthwhile considering representing identifiers using an equivalent of the FQName node.

So, I've got your idea, a solution for "input Real x[2];" could be: < fun:InputVariable name="x[2]" type="Real" > ..... or do you intend something else?

This solution would be consistent with the design of FIdUses, but requires the receiver to parse the names and extract the expressions for the array indices. Notice that these may be expressions like '2*n' in functions. This design then not consistent with the objective of providing the equivalent of an AST representation. I realize that this has more profound consequences than I thought initially.
 
Do you think that it would be feasible to introduce an extension of the schema to handle FQNames including qualified names and array subscripts? This grammar could then be used both for declarations and for identifiers.
 
Best
/Johan

jakesson
Offline
Joined: 2009-03-14
For indices
rparrotto wrote:

currently the INDEX of the for loop is designed like this:
< fun:Index IterationVariable = "i" >
< fun:IterationSet>
< exp: an expression >....
< /fun:IterationSet>
< /fun:Index>
for example the code
-------------------------
for i in {1,5} loop
end loop;
------------------------
produces
< fun:For>
< fun:Index IterationVariable="i">
< fun:IterationSet>
< exp:Array>
< exp:IntegerLiteral>1< /exp:IntegerLiteral>
< exp:IntegerLiteral>5< /exp:IntegerLiteral>
< /exp:Array>
< /fun:IterationSet>
< /fun:Index>
< fun:Statements>....
But how loops like
for i in 1:4 loop
end loop;
are represented in the AST ? I generate the code from:
FForIndex : CommonForIndex ::= FVariable;
abstract CommonForIndex ::= [FExp];

Expressions like '1:4' and ' 1:2:3' are called FRangeExp and is part of the flat Modelica grammar. Representing this type of for indices is then just a matter of supporting this type of expression.
 
Best
/Johan
 

jmattsson
Offline
Joined: 2009-10-18
FIdUse.name() in functions
rparrotto wrote:

3) VARIABLES INITIAL VALUES. The code generated for "output Real y[2]={28,91};" is just:

< fun:OutputVariable name="y" type="Real[2]" xsi:nil="true"/>

where nil="true" means that there is no initial value. It seems like there is no initial value represented in the AST, but the binding of the variable and initial values (y[1]:=28, y[2]:=91) are made at the beginning of the algorithm section as assignment nodes. It is perhaps logically correct, but I'm guessing if it is the result you wanted while designing the AST.

All binding expressions are transformed to assignments during scalarization to simplify code generation.

rparrotto wrote:

4) FLATTENED VARIABLES. Representing the assignment y:=x in the algorithm, the result I have is [...]
So the reference to the index of the array generating the variable is missing. I used FIdUse.name() to get the name of the variable (and I have the correct result when they are used in equations). What's wrong?

Since functions handle arrays as arrays, not individual variables, the scalarized flag is not set for FQNames in functions. This is required for name lookup, etc. Use toString() or prettyPrint("") instead, and it should be work.

jmattsson
Offline
Joined: 2009-10-18
nil
rparrotto wrote:

The element OutputVariable (such as Input and Protected) are defined by an expression and attributes name and type, like this:

< fun:OutputVariable name="name" type="Real" >
< exp:RealLiteral>1.2< /exp:RealLiteral>
< /fun:OutputVariable>

if the expression doesn't exist, then the element has nil value (notice that this attribute is not defined in the schema, but it is proper of XML schema language), represented by that attribute. I've used it to have a more compact code, because this allows to have the short tag representation (one line, as < element /> instead of < element>< /element>:

< fun:OutputVariable name="name" type="Real" nil="true"/>

Isn't the nil attribute implicit in the lack of an expression in the fun:OutputVariable tag? That is, if there is no expression in the variable tag, then there is no binding expression. That is the usual way of handling optional children in XML, isn't it?

rparrotto
Offline
Joined: 2009-10-01
variables' name
jmattsson wrote:

Since functions handle arrays as arrays, not individual variables, the scalarized flag is not set for FQNames in functions. This is required for name lookup, etc. Use toString() or prettyPrint("") instead, and it should be work.

Solved, thank you

rparrotto
Offline
Joined: 2009-10-01
Binding exp for variables
jmattsson wrote:
rparrotto wrote:

The element OutputVariable (such as Input and Protected) are defined by an expression and attributes name and type, like this:

< fun:OutputVariable name="name" type="Real" >
< exp:RealLiteral>1.2< /exp:RealLiteral>
< /fun:OutputVariable>

if the expression doesn't exist, then the element has nil value (notice that this attribute is not defined in the schema, but it is proper of XML schema language), represented by that attribute. I've used it to have a more compact code, because this allows to have the short tag representation (one line, as < element /> instead of < element>< /element>:

< fun:OutputVariable name="name" type="Real" nil="true"/>

Isn't the nil attribute implicit in the lack of an expression in the fun:OutputVariable tag? That is, if there is no expression in the variable tag, then there is no binding expression. That is the usual way of handling optional children in XML, isn't it?

It can be implicit if you are talking about XML language (document well-formed), not XML Schema language. If an element is designed as "nillable" in the schema, it must have the attribute nil if it has no value, otherwise it isn´t validated. Notice the expression is NOT an element inside the OutputVariable element, but it is the content of the variable itself (i.e. the variable is defined as expression + some attributes). The alternative is to create an optional expression element, say "BindingExpression", inside OutputVariable

jmattsson
Offline
Joined: 2009-10-18
nil
rparrotto wrote:

It can be implicit if you are talking about XML language (document well-formed), not XML Schema language. If an element is designed as "nillable" in the schema, it must have the attribute nil if it has no value, otherwise it isn´t validated. Notice the expression is NOT an element inside the OutputVariable element, but it is the content of the variable itself (i.e. the variable is defined as expression + some attributes). The alternative is to create an optional expression element, say "BindingExpression", inside OutputVariable

How about having a special content type for default values that can either be empty or contain an expression?

rparrotto
Offline
Joined: 2009-10-01
null expressions
jmattsson wrote:

How about having a special content type for default values that can either be empty or contain an expression?

It could be. Or another solution can be to add to the schema a new expression exp:EmptyExp ... What do you think? Johan?

rparrotto
Offline
Joined: 2009-10-01
Left side of function calls

1f) The function call statement is defined like this:

FFunctionCallStmt : FStatement ::= Left:FFunctionCallLeft* Call:FAbstractFunctionCall;

FFunctionCallLeft ::= [FExp];

but I noticed that if the left part is composed by more expressions, they are enclosed in only one vector expression: is it correct? If it is, why Left is defined as a list of FFunctionCallLeft?

2f) currently the left part is represented in the schema by an element of string type (that is supposed to be the identifier of the variable). Is it better to have it as expressions? As an array? Or would it be too ambiguous? This modify can be applied not only in the function call statements (inside the algorithms of the functions), but also to the equations produced by function calls.

3f) always regarding the left part, maybe having an empty expression could be useful in this case to represent cases like (a, ,c) = f(x)

jmattsson
Offline
Joined: 2009-10-18
Left side of function calls

The left side of a FFunctionCallStmt is a list of FFunctionCallLeft because of cases like (a, ,c) = f(x). To start with, note that this is not a vector expression, it is a list of variables receiving return values from the function. The reason for having a special node type for this is that the variable in each slot is optional, but the slot still has to be counted. In the example the second slot is empty, and the corresponding FFunctionCallLeft has no FExp.

In the scalarized model, equations with function calls that return arrays or records are represented as FFunctionCallEquations where the corresponding FExp is an FArray or FRecordConstructor. Thus, the child of an FFunctionCallLeft has to be an FExp.

rparrotto
Offline
Joined: 2009-10-01
left part of function call equations
jmattsson wrote:

In the scalarized model, equations with function calls that return arrays or records are represented as FFunctionCallEquations where the corresponding FExp is an FArray or FRecordConstructor. Thus, the child of an FFunctionCallLeft has to be an FExp.

As you correctly said "To start with, note that this is not a vector expression, it is a list of variables receiving return values from the function". It is exactly for this reason that it is not necessary (and I think it is wrong) to represent the returned value as an Exp. If you say so it is better a list of < exp:Identifier> or merely a list of elements of type string representing the resulting variable.

If we don't want to have equations (FFunctionCall) like x+2 = f(x) or sin(x) = f(x) it's more clear to have just a list of identifiers.

Notice, Jesper, that my goal is not to export documents and have schema in a way that bestly fit with JModelica's grammar, but that is most general as possible, so i think it's better to avoid VALID documents with the previous kind of equations.

jmattsson
Offline
Joined: 2009-10-18
left part of function call equations

Consider the following model:

model M
function F
output Real o[2] := {1,2};
algorithm
end F;

Real x[2] = F();
end M;

It will, after flattening and transformations be represented as:

fclass M
Real x[1];
Real x[2];
equation
({x[1], x[2]}) = M.F();

function M.F
output Real o[2];
algorithm
o[1] := 1;
o[2] := 2;
end M.F;
end M;

Here, the FExp of the (single) FFunctionCallLeft is an FArray. Thus a single identifier will not work. If you want to represent the allowed values exactly, they are identifiers, FArrays containing allowed values and record constrictors with allowed values as arguments. (See RecordTests.mo for examples.)

rparrotto
Offline
Joined: 2009-10-01
counterexample
jmattsson wrote:

Here, the FExp of the (single) FFunctionCallLeft is an FArray. Thus a single identifier will not work. If you want to represent the allowed values exactly, they are identifiers, FArrays containing allowed values and record constrictors with allowed values as arguments. (See RecordTests.mo for examples.)

In fact I talked about a list of identifiers, not of a "single identifier" and your model is not a counterexample for my solution, but the contrary.

< FunctionCallEquation>
< OutputVariable> x[1] < /OutputVariable>
< OutputVariable> x[2] < /OutputVariable>
< SourceFunction name = "M.F">
< Arguments>< < /Arguments>
< /SourceFunction>
< /FunctionCallEquation>

Where this has to be considered as 2 different equations, one for each output variable.

PS: notice that there is no record support in my project.

rparrotto
Offline
Joined: 2009-10-01
QNames
jakesson wrote:
rparrotto wrote:
jakesson wrote:

2) I suggest you represent the variable type with one attribute type="Real" and then the array subscripts as a list (containing 0 or more entries) of expressions. This separation will probably make it easier to handle the functions when doing code generation. On the other hand, this is the same issue as how to represent array indices in identifiers. It may be worthwhile considering representing identifiers using an equivalent of the FQName node.

So, I've got your idea, a solution for "input Real x[2];" could be: < fun:InputVariable name="x[2]" type="Real" > ..... or do you intend something else?

This solution would be consistent with the design of FIdUses, but requires the receiver to parse the names and extract the expressions for the array indices. Notice that these may be expressions like '2*n' in functions. This design then not consistent with the objective of providing the equivalent of an AST representation. I realize that this has more profound consequences than I thought initially.
 
Do you think that it would be feasible to introduce an extension of the schema to handle FQNames including qualified names and array subscripts? This grammar could then be used both for declarations and for identifiers.
 
Best
/Johan

I see your point and it is a tricky problem. I can try to find a solution for the representation of names, but for sure it will complicate the schema.

A question: is this kind of notation necessary also in the equation part or just in the algorithms? I mean, when we are using variables in the equation section, the index has no reason to be evaluated as an expression, right?

jakesson
Offline
Joined: 2009-03-14
nil
rparrotto wrote:
jmattsson wrote:

How about having a special content type for default values that can either be empty or contain an expression?

It could be. Or another solution can be to add to the schema a new expression exp:EmptyExp ... What do you think? Johan?

This issue is closely related to how binding expressions are represented in parameter declarations: what mechanism is used there? Preferably we should use the same mechanism here. In any case, may opinion is that representing the binding expression as an element <BindingExpression> makes the schema a bit clearer and I would therefore prefer this solution.
 
Best
/Johan

jakesson
Offline
Joined: 2009-03-14
FQName
rparrotto wrote:
jakesson wrote:
rparrotto wrote:
jakesson wrote:

2) I suggest you represent the variable type with one attribute type="Real" and then the array subscripts as a list (containing 0 or more entries) of expressions. This separation will probably make it easier to handle the functions when doing code generation. On the other hand, this is the same issue as how to represent array indices in identifiers. It may be worthwhile considering representing identifiers using an equivalent of the FQName node.

So, I've got your idea, a solution for "input Real x[2];" could be: < fun:InputVariable name="x[2]" type="Real" > ..... or do you intend something else?

This solution would be consistent with the design of FIdUses, but requires the receiver to parse the names and extract the expressions for the array indices. Notice that these may be expressions like '2*n' in functions. This design then not consistent with the objective of providing the equivalent of an AST representation. I realize that this has more profound consequences than I thought initially.
 
Do you think that it would be feasible to introduce an extension of the schema to handle FQNames including qualified names and array subscripts? This grammar could then be used both for declarations and for identifiers.
 
Best
/Johan

I see your point and it is a tricky problem. I can try to find a solution for the representation of names, but for sure it will complicate the schema.

Is it really a tricky thing: I mean, the grammar for FQName, FArraySubscripts and FSubscript is pretty straightforward, right?

Quote:

A question: is this kind of notation necessary also in the equation part or just in the algorithms? I mean, when we are using variables in the equation section, the index has no reason to be evaluated as an expression, right?

If we go ahead and define this extension, then we should use it also in equation sections, I think. This would greatly simplify translation of array names into various languages, since a name like 'x[2]' then would not have to be parsed, the structured AST representation could be used instead.
 
Best
/Johan
 

jakesson
Offline
Joined: 2009-03-14
Output arguments
rparrotto wrote:
jmattsson wrote:

Here, the FExp of the (single) FFunctionCallLeft is an FArray. Thus a single identifier will not work. If you want to represent the allowed values exactly, they are identifiers, FArrays containing allowed values and record constrictors with allowed values as arguments. (See RecordTests.mo for examples.)

In fact I talked about a list of identifiers, not of a "single identifier" and your model is not a counterexample for my solution, but the contrary.
< FunctionCallEquation>
< OutputVariable> x[1] < /OutputVariable>
< OutputVariable> x[2] < /OutputVariable>
< SourceFunction name = "M.F">
< Arguments>< < /Arguments>
< /SourceFunction>
< /FunctionCallEquation>
Where this has to be considered as 2 different equations, one for each output variable. PS: notice that there is no record support in my project.

It seems to me that this solution lacks some of the expressivity that is really needed. For example, the output arguments '({x[1],x[2]}, ,{y[1],y[2]})' are not the same as '(x[1],x[2], ,y[1],y[2])', that is, the grouping matters: in the first case, the function has three output arguments, whereas in the second case it has five output arguments. I don't realize how these two different cases would be represented in a way that they can be distinguished? 
 
One way of still having type checking would be to introduce a new node ArrayOfIdentifiers and require that an output variable is either an identifier or an ArrayOfIdentifiers? 
 
Best
/Johan

rparrotto
Offline
Joined: 2009-10-01
binding exp
jakesson wrote:
rparrotto wrote:
jmattsson wrote:

How about having a special content type for default values that can either be empty or contain an expression?

It could be. Or another solution can be to add to the schema a new expression exp:EmptyExp ... What do you think? Johan?

This issue is closely related to how binding expressions are represented in parameter declarations: what mechanism is used there? Preferably we should use the same mechanism here. In any case, may opinion is that representing the binding expression as an element <BindingExpression> makes the schema a bit clearer and I would therefore prefer this solution.
 
Best
/Johan

They are represented by another element "BindingExpression" for the parameters. Yes, we should use the same mechanism. I've just tried to limit the number of nodes, but you're right, maybe this is still the preferable solution.

rparrotto
Offline
Joined: 2009-10-01
grouping
jakesson wrote:
rparrotto wrote:
jmattsson wrote:

Here, the FExp of the (single) FFunctionCallLeft is an FArray. Thus a single identifier will not work. If you want to represent the allowed values exactly, they are identifiers, FArrays containing allowed values and record constrictors with allowed values as arguments. (See RecordTests.mo for examples.)

In fact I talked about a list of identifiers, not of a "single identifier" and your model is not a counterexample for my solution, but the contrary.
< FunctionCallEquation>
< OutputVariable> x[1] < /OutputVariable>
< OutputVariable> x[2] < /OutputVariable>
< SourceFunction name = "M.F">
< Arguments>< < /Arguments>
< /SourceFunction>
< /FunctionCallEquation>
Where this has to be considered as 2 different equations, one for each output variable. PS: notice that there is no record support in my project.

It seems to me that this solution lacks some of the expressivity that is really needed. For example, the output arguments '({x[1],x[2]}, ,{y[1],y[2]})' are not the same as '(x[1],x[2], ,y[1],y[2])', that is, the grouping matters: in the first case, the function has three output arguments, whereas in the second case it has five output arguments. I don't realize how these two different cases would be represented in a way that they can be distinguished? 
 
One way of still having type checking would be to introduce a new node ArrayOfIdentifiers and require that an output variable is either an identifier or an ArrayOfIdentifiers? 
 
Best
/Johan

Yes, you're right. I didn't think about this "grouping problem". Yes, the idea to have either identifier or array of identifiers is feasible and better than allowing more general expressions.

At this point I'd add a third possible choice, to represent the empty outuput arguments, say < EmptyOutputArgument>

jmattsson
Offline
Joined: 2009-10-18
recusion
rparrotto wrote:

Yes, you're right. I didn't think about this "grouping problem". Yes, the idea to have either identifier or array of identifiers is feasible and better than allowing more general expressions.

At this point I'd add a third possible choice, to represent the empty outuput arguments, say < EmptyOutputArgument>

Don't forget that the arrays should be recursive, e.g. {{{x[1,1,1],x[1,1,2]}}} is allowed. Also remember that < EmptyOutputArgument> shouldn't be allowed in an array (since you will need a recursive definition).

I'd suggest having a "left" that can contain an "exp" or "empty", where "exp" is an "identifier" or an "array" and an "array" is a list of "exp" (with better names, of course).

rparrotto
Offline
Joined: 2009-10-01
vectors of identifiers
jmattsson wrote:

Don't forget that the arrays should be recursive, e.g. {{{x[1,1,1],x[1,1,2]}}} is allowed...

Good observation, but I don't think they necessarily should be allowed: it is always possible to produce just one long array from an array of array. It's a restriction that could be applied on the scalarization algorithm and a matter of design. How has it been implemented in JModelica? Johan, what do you think about this: should we or should we not have recursive array of identifiers? (it is not a problem to design in this in the schema, anyway, but just a choice). In my opinion having just one array allows an easier import, especially of the FunctionCallEquations

jmattsson
Offline
Joined: 2009-10-18
recursive arrays

The recursive arrays are there because the types Real[2,2] and Real[4] are not the same...

rparrotto
Offline
Joined: 2009-10-01
recursive arrays
jmattsson wrote:

The recursive arrays are there because the types Real[2,2] and Real[4] are not the same...

It's because you are thinking to the Modelica representation, while my schema is the representation of a mathematical model, a DAE, not of a Modelica model. Real[2,2] and Real[4] produce a set of the same 4 equations, but the second one (without recursion)is a more clear representation for the equations set and it's easier to import and understand. It's always better to avoid more complex data structures in this case, because the meaning is always a SET of equations like scalarvariable = functioncall....

It is possible to differentiate in my schema the function calls relative to the FunctionCallEquations and the ones inside the algorithms of user-defined functions, allowing this kind of recursion only in the second case. I think it's a better way.

jmattsson
Offline
Joined: 2009-10-18
recursive arrays

The problem is that while a multi-dimensional array can be represented as a single-dimensional one, the function definitions handles them as multi-dimensional. Another problem is that a 2-dimensional array can be converted to a single-dimensional one in two common ways - row-major and column-major. How do you plan to encode that information? Also, it will be illogical for a multidimensional array to be flattened out into a single-dimensional one there and only there.

jakesson
Offline
Joined: 2009-03-14
Multi dimensional array outputs
jmattsson wrote:

The problem is that while a multi-dimensional array can be represented as a single-dimensional one, the function definitions handles them as multi-dimensional. Another problem is that a 2-dimensional array can be converted to a single-dimensional one in two common ways - row-major and column-major. How do you plan to encode that information? Also, it will be illogical for a multidimensional array to be flattened out into a single-dimensional one there and only there.

First of all, we already have (multi-dimensional) arrays in functions in the schema, so allowing them in function output arguments does not add a new kind of expression. Also, multi-dimensional arrays are not Modelica-specific objects, but rather generic mathematical ones, so I see no problem with having them in the schema for the sake of generality.
 
The problem with using one-dimensional arrays to represent multi-dimensional ones is, as Jesper points out, that the mapping between a multi-dimensional array function output argument and a one-dimensional counterpart amongst the output arguments in an EquationFunctionCall is ambiguous. In an early design, we figured that functions could be fully scalarized, and then it would have been ok to "flatten" arrays, since then the output arguments of the function would have been ordered. But since we opted for a design where arrays are preserved in functions, it follows as a consequence that we need to represent output arguments as multi-dimensional arrays as well.
 
Best
/Johan 

rparrotto
Offline
Joined: 2009-10-01
still on outputs
jakesson wrote:

The problem with using one-dimensional arrays to represent multi-dimensional ones is, as Jesper points out, that the mapping between a multi-dimensional array function output argument and a one-dimensional counterpart amongst the output arguments in an EquationFunctionCall is ambiguous. In an early design, we figured that functions could be fully scalarized, and then it would have been ok to "flatten" arrays, since then the output arguments of the function would have been ordered. But since we opted for a design where arrays are preserved in functions, it follows as a consequence that we need to represent output arguments as multi-dimensional arrays as well.
 
Best
/Johan 

Yes, that's the source of the misunderstanding between me and Jesper. At the beginning of the project we supposed full scalarization and ordering of the variables. I see your point now, but I need the feedback of prof. Casella on this, because he doesn't agree I will never have my master :)

jakesson
Offline
Joined: 2009-03-14
Output arguments
rparrotto wrote:
jakesson wrote:

The problem with using one-dimensional arrays to represent multi-dimensional ones is, as Jesper points out, that the mapping between a multi-dimensional array function output argument and a one-dimensional counterpart amongst the output arguments in an EquationFunctionCall is ambiguous. In an early design, we figured that functions could be fully scalarized, and then it would have been ok to "flatten" arrays, since then the output arguments of the function would have been ordered. But since we opted for a design where arrays are preserved in functions, it follows as a consequence that we need to represent output arguments as multi-dimensional arrays as well.
 
Best
/Johan 

Yes, that's the source of the misunderstanding between me and Jesper. At the beginning of the project we supposed full scalarization and ordering of the variables. I see your point now, but I need the feedback of prof. Casella on this, because he doesn't agree I will never have my master :)

Let's hope that Francesco agrees :). 
 
Best
/Johan 

Login or register to post comments