TOPIC:

Hi All,

I found that activate operation is not possible without an assignment of result value to variable.

I.e.:

activate(DefaultStep);

is not possible, always should be in a form of:

MyDefaultVariable := activate(DefaultStep);

That is handled in
Primary ::= OpCall | Value | "(" SingleExpression ")"

through OpCall definition.

In most test cases, where only one default is activated there may be no need for default deactivation and no need to store handle of activated default.
That's I propose to modify definition below to include an ActivateOp definition:

BehaviourStatements ::= TestcaseInstance |

FunctionInstance |

ReturnStatement |

AltConstruct |

InterleavedConstruct |

LabelStatement |

GotoStatement |

RepeatStatement |

DeactivateStatement |

AltstepInstance |

ActivateOp



This solution is similar to execute statement /an operation returning result, which also does not need to be assigned/ which is through TestcaseInstance definition present twice: in BehaviourStatements and in OpCall.

BR,
Mariusz Kupiec

Hi,

This makes sense, (need for a handle) since there would be no way to deactivate the activated default.

If your proposal of 'deactivate' implicitly referencing all activated defaults in a given component is accepted
then there might be a need for this. Otherwise I believe that forcing the usage of a default handle will hopefully result
in easier to understand code.

One of the initial design objectives of TTCN-3 was to eliminate/reduce implicit behaviour. In other words, nothing happens
unless there is some code which explicitly says that something should happen.

E.g. in TTCN-2, each time an activate statement is used, all currently activated defaults are deactivated, in TTCN-3 all defaults
must be explicitly deactivated. Since the code is visible, one knows what is happening. This is particularly useful for those
new to TTCN-3.

One could argue that the implicit behaviour reduces the amount of code and thus readability, but on the downside it can create
some learning problems for those unaware that some behaviour is occuring implicitly and lead them to misunderstand what is
actually happening. :-).

Just my two pence...

Cheers,

Claude.

ps: see below also.

---

Original Message

---

From: Mariusz Kupiec [This email address is being protected from spambots. You need JavaScript enabled to view it.]
Sent: 17 December 2002 16:18
To: This email address is being protected from spambots. You need JavaScript enabled to view it.
Subject: Activate operation without mandatory assignment of result


Hi All,

I found that activate operation is not possible without an assignment of result value to variable.

I.e.:

activate(DefaultStep);

is not possible, always should be in a form of:

MyDefaultVariable := activate(DefaultStep);

That is handled in

Primary ::= OpCall | Value | "(" SingleExpression ")"

through OpCall definition.

In most test cases, where only one default is activated there may be no need for default deactivation and no need to store handle of activated default.

Well, this depends on which test suites you have looked at, worked with or developed. One could also generalize that many test cases do not use
any defaults, or multiple defaults. I feel that not deactivating a default makes for sloppy code.


That's I propose to modify definition below to include an ActivateOp definition:


BehaviourStatements ::= TestcaseInstance |

FunctionInstance |

ReturnStatement |

AltConstruct |

InterleavedConstruct |

LabelStatement |

GotoStatement |

RepeatStatement |

DeactivateStatement |

AltstepInstance |

ActivateOp




This solution is similar to execute statement /an operation returning result, which also does not need to be assigned/ which is through TestcaseInstance definition present twice: in BehaviourStatements and in OpCall.

BR,
Mariusz Kupiec

---

WIADOMOSCI minuta po minucie... >>> link.interia.pl/f16a8

Hi Claude,

None generalization is acceptable here :-)
All proposals are result of observations and yet another try of code simplification.

Test system makes many operations implicitly and can do one more operation while MTC terminates.
Moreover, proposal of activate in 'short version' is only an alternative for complete operation, like in similar case of execute operation.

EXAMPLE of other test system implicit behaviour:

One could easily find that although execute returns verdict, but half of examples in the spec do not use it. Why?

The question may be:
- verdicts are implicitly redirected to output device (display screen?),
- verdicts are implicitly logged (in tester buffer?),
- verdicts are explicitly skipped, neither stored, neither logged, because of no importance of execute operation in example (something other stressed in example).

The two first are under implicit assumption that not only execute operation itself but rather test system and integrated environment is responsible for handling verdicts. Another argument for this assumption is that no one of examples uses verdict logging.

That leads to another question.
If test suite should be portable then how verdicts should be handled?
Should be stressed that verdicts are always accessible via test system (like in TTCN-2) which collects all information about test cases execution?
This is good point of view because final test case verdit is a part of MTC information record and MTC lifetime is only during test case execution so verdict should be stored somewhere in a test system.
It looks like returning verdict from an execute operation is only nice add-on and verdicts returned by execute operation, if not stored are not really lost :-)
Those unaware may be sure everything was lost :-)
If test system collects verdicts from test cases why not to give access to a final verdict of every test case in a form of e.g. MyTestCase.verdict ?
It would be accessible before, during and after test case execution without trouble of storing in variables.

What is meaning of operation setverdict(none)?
Does it mean 'something happen but nothing interesting' for us ? :-)
What's the purpose of setting this kind of verdict?
Shouldn't it be used for only initialization purposes by test system only and as a return information that no verdict was issued?

What is benefit from using an error verdict only as a final test case value?
If a runtime error happens due to test system or test suite limitations/design error /e.g. overflow, no initialization of variables/ than execution of test case should be stopped immediately and an error verdict as soon as possible should help to identify location of an error?
That is what I am guessing now :-)
It is not written explicitly that after an error verdict issued by at least one PTC the system should stop test case execution.
But it is written that getverdict never returns an error verdict.
I see three possibilities:
- test case is stopped immediately with an error verdict;
- test case is not stopped, an error verdict is also accessible through getverdict as a local component verdict;
- test case is not stopped, an error verdict is not accessible until test case finished.

The last is wrong and would be a waste of time, the middle is not allowed, the first is what I expect but it is not explicitly written.


Cheers,
Mariusz



Claude

---

Original Message

---

From: Claude Desroches
To: This email address is being protected from spambots. You need JavaScript enabled to view it.
Sent: Wednesday, December 18, 2002 10:44 AM
Subject: Re: Activate operation without mandatory assignment of result



Hi,

This makes sense, (need for a handle) since there would be no way to deactivate the activated default.

If your proposal of 'deactivate' implicitly referencing all activated defaults in a given component is accepted
then there might be a need for this. Otherwise I believe that forcing the usage of a default handle will hopefully result
in easier to understand code.

One of the initial design objectives of TTCN-3 was to eliminate/reduce implicit behaviour. In other words, nothing happens
unless there is some code which explicitly says that something should happen.

E.g. in TTCN-2, each time an activate statement is used, all currently activated defaults are deactivated, in TTCN-3 all defaults
must be explicitly deactivated. Since the code is visible, one knows what is happening. This is particularly useful for those
new to TTCN-3.

One could argue that the implicit behaviour reduces the amount of code and thus readability, but on the downside it can create
some learning problems for those unaware that some behaviour is occuring implicitly and lead them to misunderstand what is
actually happening. :-).

Just my two pence...

Cheers,

Claude.

ps: see below also.

---

Original Message

---

From: Mariusz Kupiec [This email address is being protected from spambots. You need JavaScript enabled to view it.]
Sent: 17 December 2002 16:18
To: This email address is being protected from spambots. You need JavaScript enabled to view it.
Subject: Activate operation without mandatory assignment of result


Hi All,

I found that activate operation is not possible without an assignment of result value to variable.

I.e.:

activate(DefaultStep);

is not possible, always should be in a form of:

MyDefaultVariable := activate(DefaultStep);

That is handled in
Primary ::= OpCall | Value | "(" SingleExpression ")"

through OpCall definition.

In most test cases, where only one default is activated there may be no need for default deactivation and no need to store handle of activated default.

Well, this depends on which test suites you have looked at, worked with or developed. One could also generalize that many test cases do not use
any defaults, or multiple defaults. I feel that not deactivating a default makes for sloppy code.


That's I propose to modify definition below to include an ActivateOp definition:

BehaviourStatements ::= TestcaseInstance |

FunctionInstance |

ReturnStatement |

AltConstruct |

InterleavedConstruct |

LabelStatement |

GotoStatement |

RepeatStatement |

DeactivateStatement |

AltstepInstance |

ActivateOp



This solution is similar to execute statement /an operation returning result, which also does not need to be assigned/ which is through TestcaseInstance definition present twice: in BehaviourStatements and in OpCall.

BR,
Mariusz Kupiec

---

WIADOMOSCI minuta po minucie... >>> link.interia.pl/f16a8

---

Film, muzyka, plotki z zycia gwiazd! >>> link.interia.pl/f16a9

In einer eMail vom 12/18/02 1:32:44 PM W. Europe Standard Time schreibt
This email address is being protected from spambots. You need JavaScript enabled to view it.:

Hi Again,

>
> Hi Claude,
>
> None generalization is acceptable here :-)
>
Agreed.

All proposals are result of observations and yet another try of code
simplification.
>
>
> Test system makes many operations implicitly and can do one more operation
> while MTC terminates.
> Moreover, proposal of activate in 'short version' is only an alternative
> for complete operation, like in similar case of execute operation.
>
>
> EXAMPLE of other test system implicit behaviour:
>
> One could easily find that although execute returns verdict, but half of
> examples in the spec do not use it. Why?
>

I suspect that this is an oversight. I do not know what should happen with
return values
from operations, functions, or other where no assignment is made. I would
assume
as many others would that this value is simply discarded. Use C as an
analogy.

What happens with the return value of a C function which returns an integer?
The
return value is discarded/ignored.

Maybe this should be spellt out explicitly in the standard to avoid confusion
and
misinterpretation, or even worse differing implemenations with respect to
behaviour!

>
> The question may be:
> - verdicts are implicitly redirected to output device (display screen?),
> - verdicts are implicitly logged (in tester buffer?),
>

As a reminder, TTCN-3 has attempted to avoid any type of 'implicit'
behaviour. Nothing
should happen implicitly (there are pragmatic exceptions such as implicit
creation of the
MTC by the test system, not needing to specify the system component if only
the
MTC is used, etc.


> - verdicts are explicitly skipped, neither stored, neither logged, because
> of no importance of execute operation in example (something other stressed
> in example).
>
>
> The two first are under implicit assumption that not only execute operation
> itself but rather test system and integrated environment is responsible for
> handling verdicts. Another argument for this assumption is that no one of
> examples uses verdict logging.
>
Well, this is one of the differences, an important one, between TTCN-2 and
TTCN-3.
Final Verdicts in TTCN-2 are assigned by the MTC, whereas in TTCN-3 they are
assigned by the test system.

Because TTCN-3 offers the log statement, the implicit logging of verdicts is
felt to
be superfluous, now if you wanted to mimick the TTCN-2 operational semantics
in
TTCN-3 you would need to look at the TTCN-2 annex describing the TTCN-2
operational
semantics. I'm assuming here that TTCN-3 operational semantics don't force
the
logging of verdicts (need to check this).


>
> That leads to another question.
> If test suite should be portable then how verdicts should be handled?
>
Verdicts are none, pass, fail, inconc, and error. The test system is
responsible for
managing these. So long as the verdict semantics are respected by a test
system,
how they are implemented by a system is irrelevant I believe. Each vendor
can
choose to implement verdicts in their own way.
Standards tell you what should happen, not how. In this way, implementation
freedom is guaranteed so long as each implementation resects the semantics of
the abstract representation. :-)

Should be stressed that verdicts are always accessible via test system (like
in TTCN-2)
> which collects all information about test cases execution?

Actually, TTCN-2 does not collect verdict information beyond the test case
level. To
collect verdict info for a test campaign requires functionality outside the
scope of the
TTCN-2 standard.

In TTCN-3, the control part allows the definition of arrays which allow you
to track
verdict results obtained from test campaign execution.

> This is good point of view because final test case verdit is a part of MTC
> information record and MTC lifetime is only during test case execution so
> verdict should be stored somewhere in a test system.
>
Actually, in TTCN-3 the MTC only maintains the local MTC component verdict.
Not
the global test case verdict result. TTCN-2 and TTCN-3 differ on this point.
In TTCN-2
the MTC maintains the final verdict result. In TTCN-3 the test system
maintains the
final verdict, not the MTC, since as stated in the TTCN-3 standard, that
get/setverdict
have local scope. (I'm not sure that I've followed what you are saying above,
sorry
if I've misinterpreted)...

It looks like returning verdict from an execute operation is only nice add-on
and verdicts
> returned by execute operation, if not stored are not really lost :-)
> Those unaware may be sure everything was lost :-)
> If test system collects verdicts from test cases why not to give access to
> a final verdict of every test case in a form of e.g. MyTestCase.verdict ?
> It would be accessible before, during and after test case execution without
> trouble of storing in variables.
>
Now that is another good suggestion!

>
> What is meaning of operation setverdict(none)?
>
It means that you are ensuring that the local verdict of a given component is
being
set to none. Since verdicts of all components are 'implicitly' set to none
when they
are started (I'm assuming) such an assignment would be redundant.


> Does it mean 'something happen but nothing interesting' for us ? :-)
> What's the purpose of setting this kind of verdict?
> Shouldn't it be used for only initialization purposes by test system only
> and as a return information that no verdict was issued?
>
Returning a verdict of none as usual should result in a test case error,
since no
conclusion can be made about the execution of a test case. It indicates that
the test case is incompletely specified.

What is the point of executing a test case? To obtain a verdict which
provides further
information regarding the behaviour of a system being tested. So, if the
verdict is
none, it would be the same as if we had not executed the test case. That is,
no
new information has been provided (well, if you log information, then you
will have that)
including the none verdict...

>
> What is benefit from using an error verdict only as a final test case
> value?

Well, I assume here that if an error verdict occurs, (which can only be set
by the test
system) that the test case execution will halt. (it should)

> If a runtime error happens due to test system or test suite
> limitations/design error /e.g. overflow, no initialization of variables/
> than execution of test case should be stopped immediately and an error
> verdict as soon as possible should help to identify location of an error?
> That is what I am guessing now :-)

Me too.

> It is not written explicitly that after an error verdict issued by at least
> one PTC the system should stop test case execution.

Well, the test system returns the error verdict, execution of a given PTC may
cause the
error verdict to happen, but the PTC itself will not return an error verdict,
since the PTC
is part of the test case.

The test system assigns the final verdict. A test case itself can only
return, none,
pass, fail, inconc.

But it is written that getverdict never returns an error verdict.
>
> I see three possibilities:
> - test case is stopped immediately with an error verdict;
> - test case is not stopped, an error verdict is also accessible through
> getverdict as a local component verdict;
> - test case is not stopped, an error verdict is not accessible until test
> case finished.
>
> The last is wrong and would be a waste of time, the middle is not allowed,
> the first is what I expect but it is not explicitly written.
>
>
>
> Cheers,
> Mariusz
>
>
>
> Cheers,


Claude