Concurrency and atomicity questions

[Slapstick]

Hi all,

I'm a long time OFP player just getting back into the game after a time away (getting psyched for ArmA).  Previously, I had no interest in mission design or scripting, this time around I'm finding it one of the more enjoyable aspects of the game!  Lately, I've been fiddling around with concurrency issues, that is, having multiple versions of the same script running at the same time and I have a few questions/observations.

I hope this question is sufficiently advanced; I've always considered multi-tasking and concurrency to be relatively advanced topics!

First, does anyone know what the basic "atomic" unit is in an OFP script?  That is, what is the smallest "unit" that will be executed where you are guaranteed that execution won't be interupted by another script?  I've done some experimenting and it seems that a "line" (one or more statements separated by semi-colon on a single line) forms the basic "atomic" unit in OFP scripts.  Can anyone confirm/deny this?  I've looked through the forums abit, but haven't seen anything on this topic.

Secondly, on a related topic, I haven't experimented much with functions, but I've read that long running functions can have a detrimental effect on game performance.  This would lead me to believe that each function forms an atomic unit as well, that is, a function will never be preempted, and that functions aren't multitasked the same way that scripts are.  Again, can anyone confirm or deny this?

Finally, I suspect the answer to my next question is no, but thought I would ask anyway.  Does OFP provide any mechanisms for dealing with concurrency issues (mutexes, locks, etc.)?

Thanks in advance

[Mikero]

hi :wave:

there are much cleverer people here than me on this subject, but, in my own research, the basic atomic unit, in the way you describe it, is indeed a 'line', as opposed to any individual commands that might make it up.

a 'line' in this definition is what the engine will parse things out to which obviously means comments and #defines or other beasties are stripped out thus there's rarely a physical correlation. Basically

anything[]={"thing", thang, thongle()};

is a 'line' and atomic in your definition.

as would be

this \
that \
the other;


be aware that

#define thang "some very large text"

remains atomic. the entire macro is processed, 'atomicly', within the same 'atomic' constuct it is encountered in.

also, you might not be aware that there is NO binary compile in ofp. Text is text is text is scripts. The engine interprets syntax as it encounters it.

One other trap for unwary players is that looping scripts cannot be altered once they begin. eg, altering the .sqs file from which they derive, has no effect, as the script itself remains resident in the ofp engine (or its save.fps file) until the loop exits.

You're into interesting territory

[THobson]

Not surprisoing that Mikro was the first to reply.  I only suspevt what you say is true, I have done no testing so I will believe you.

Does OFP provide any mechanisms for dealing with concurrency issues

Not that I have found.  Whenever I have needed something like this I have created global variables to communicate what I need to have communicated.

[Mikero]

I've read that long running functions can have a detrimental effect on game performance.  This would lead me to believe that each function forms an atomic unit as well, that is, a function will never be preempted, and that functions aren't multitasked the same way that scripts are.  Again, can anyone confirm or deny this?

 'detrimental' simply means that if you wanted to respond or do other thngs 'at the same time' you can't.  Functions (.sqf) files are un-interruptible. They run, until they exit.

there are specific commands that cannot be used in functions for this reason. The engine sets a timer against any executing function and aborts it if it 'runs' too long (have no idea what the timeout is though, I believe it's 0.1 secs)

But detrimental in the sense that functions somehow degrade a mission permanenly is wrong. They run, at the detriment of other things, including 'triggers', which themselves are un-interruptable.

>concurrency issues

you'd have to give an example of what you were thinking of.  As Thob says, semaphore locks can be achieved by global vars, but sqs/sqf files are not re-entrant (afaik). Perhaps multiplayer scripts are where you need to be to find your answer, or, sniff around Kegety's excellent tools such as dxdll for clues about external file (message) signalling.

I'd imagine multiplayer is where you need to be, but I'm rapidly getting out of my depth and will shut up now.

[THobson]

sqs files are indeed re-entrant - unless I have an incorrect understanding of the term.  I have not tried a recursive algorithm in and ofp script, but I think is would work.

EDIT:

I think I was talking nonsense.  I was trying to think how to do it and couldn't not without creating some complex global arrays to hold the... I now realsie this is probably nonsense also.  I will think about this further.

[Spinor]

Ages ago, Marek from BIS mentioned that 100 lines in a sqs are atomic unless some waiting/jumping construct such as @,~ or goto is encountered, in which case execution immediately stops.
If you have a script without any of these and which has less than 100 lines, it should be guaranteed to run in one go. Similarly, if you have a loop with a loop body smaller than 100 lines, each iteration should also be executed in one frame.

[Slapstick]

I'll have to try a recursive script/function next to see how that works, but THobson is correct, scripts are reentrant.  For example, I've been playing around with these scripts:

Code: [Select]

;init.sqs
GV = 0

; bad.sqs
; Thread id
_id = _this select 0
; Delta value to add to the global variable, use a different
; value for each thread
_d = _this select 1
; desync the threads
~random 5
hint format ["Starting thread %1", _id]
#loop
_n = GV + _d
GV = GV + _d
? GV == _n : goto "loop"
hint format ["BOOM! Thread %1 failed", _id]

If you create two triggers, set the conditions to true, and put [1,1] exec "bad.sqs" in the onActivation field of one and [2,-1] exec "bad.sqs" in the other, one of the scripts will fail almost immediately after the other starts up.  However, the above can be made "thread safe" by either placing the body of the loop and loop test on the same line, or, as Mikero suggests, using a global variable as a semaphore.

Are there any tutorials/other threads on this subject?  Given the very subtle nature of the bugs that can be introduced by code that is not thread safe I would expect to see a couple of tutorials on the subject.  I'm sure more than a few new scripters have been baffled as to why their event handlers etc. have failed at random times.

as would be
this \
that \
the other;

Thanks, you reminded me that I wanted to ask if OFP has a line continuation character!  I assume '\' is it?  However, wouldn't one still need to separate the statements with semi-colons? i.e.

this; \
that; \
the other

They run, at the detriment of other things, including 'triggers', which themselves are un-interruptable.

Yes, that was what I was trying to ask. Thanks!

[Slapstick]

Ages ago, Marek from BIS mentioned that 100 lines in a sqs are atomic

That doesn't seem to be my finding (see above).  However, BIS may have changed the way the interpreter works to improve overall script performance since Marek posted that remark.  However, while I haven't tested it, I would assume that you are correct and any waiting/jumping construct would cause a thread to be preempted... However, my assumptions have led me into trouble with OFP before!

[Mikero]

line continuation character!  I assume '\' is it?

ok, i wont shut up.

for classname material like addons and missions, think c++ and you're mostly there.  /* */ // #include, use of ## and etc.

this; \
that;\

nope. for readability you'd tend to break it wherever semis were wanted, but not mandatory.

the semi defines the end of a token, so you get oddites like

model = a strange file somewhere; // or eol
model = "a strange file somewhere"    ; // or eol

are synonomous (the file name does indeed contain spaces)

I would expect to see a couple of tutorials on the subject(theads).

so do WE and I'm not being sarcastic suggesting you to write it.

[Dinger]

Okay, let's cut through it.

A short history:
OFP:CWC was released with a powerful scripting language that allowed the execution of
small programs stored in the form of text files. When the exec command is run, OFP loads into memory one of these textfiles (if it is not already in memory), with a limited amount of processing
OFP:R's release built upon a feature available in OFP:CWC, namely the ability of the scripting engine to execute strings, and turned this into a secondary scripting system. Now, in addition to exec'ing scripts, we can call functions, and we can load strings from files (hence call loadfile "myfunction.sqf").
--
Alright, what does this mean?

First off, as alluded to by others, there are several kinds of things you can do in OFP scripting languages. Let's classify them by their addressee, and apologies for nonstandard terminology. What you put in a script can be addressed to:

A) A Preprocessor
B) The Script Scheduler
C) The Line Processor
D) The Game Engine

Preprocessors
1) The Function Preprocessor:
__ when you call Preprocessfile, you are starting a routine that takes a string, removes all the // /* */ comments, and replaces the #defines in the text.
Instructions that target the Function Preprocessor:
//
/* */
#DEFINE
2) The Script Preprocessor:
__ exec actually runs the script file through a preprocessor before it gets into memory. The script preprocessor strips out all the ;comments and blank spaces, removes the #labels and indices them separately. It only does this once per script in memory. A script remains in memory until all instances of that script terminate. (This trivial detail is one thing easily discovered when alt-tabbing in the mission editor and editing files on the fly).
Instructions that target the Script Preprocessor:
; (comment)
#LABEL

The Script Scheduler
Well, maybe that's not what it is, but it's what it does.
OFP's engine does some fancy balancing acts between the various demands on system resources. Among other things, it dedicates time to processing all active scripts. While BIS may make claims about "atomic blocks" of up to 100 lines, claims which I wouldn't want to contradict, the scheduler allocates "time" to process scripts based on the number of lines to process. The fact that there appears to be a correlation between number of lines processed and time taken to process them does not mean that each line is processed in its own time slice.  Anyway, you can address the script scheduler with the following instructions:
~n: Suspend script for n seconds.
@condition: check script every slice for condition -- if condition obtains, continue script.
GOTO label: change the line pointer on the script to point to the line corresponding to label. (I'm not sure if a goto breaks atomicity, but ~ and @ definitely do)

EXIT: terminate the function

The Line Processor
At the heart of the OFP scripting engine is the Line Processor. This goes through the text and determines what to do. Everything on the same line is done in the same game-time slice. The following commands address the line processor:

1. Calling Functions
Functions are simply one or more OFP game engine commands that are run on the same line, that is in the same slice.

{function} forEach [array]: execute the function once for each element in the array. Particular elements are represented by _x.
{condition} Count [array]: check the condition once for each element in the array. Particular elements are represented by _x.

These two commands are OFP's oldest in terms of executing functions. The function is run in the same scope as the surrounding code; in scripts, this means that you can define local variables in a foreach/count function (I think, but I'd advise against it)

[template] call function: executes the function (a string, whether hard coded, variable (keep to less than 4096 bytes or it will be corrupted by a savegame!) or accessed via loadfile).

while {condition} do {function}: repeat function until condition obtains. Please note that this is a line processor function, so time does not pass.  while {_time < 1} do {twiddle thumbs} will never terminate.

if (condition) do {function} else {function}: the only conditional that works in a function. Opens a new scope.

Note that all these functions are run in-line. Using []exec opens a process after the line/atomic block.

2. Other stuff
Private []; in scopes below scripts (=functions), only global variables and local variables already defined in higher scopes can be called. Private defines local variables for use in that scope.
comment {}; is parsed but ignored
semicolons ( separate commands.

The Game Engine
Everything else I'll just attribute to the "game engine" more generally.


--- Particularities:
scripts and functions have slightly different syntax
In particular, scripts allow the use of ?condition:action, which acts in the same scope as the script, in addition to if (condition) then {action} else {action}, which acts in a subordinate scope.



Anyway, to answer your question: I've seen folks argue for atomicity, and I suppose it's how things work. However, I don't trust it. Anything in a function doesn't need to worry about collisions. Anything in a script needs to consider that from line to line, the game state, and everything in it changes. Time passes. If you need stuff to happen at the same time in a script, put it on the same line and separate with semicolons.

[Mikero]

@Dinger

very nice to better than that.
 

[Fragorl]

Hehe... cheers Dinger, an awesome summary there

[Killswitch]

Finally an actually advanced thread in here  ;D The subjects of atomicity and synchronisation are indeed blurry in OFP. Me, I use this "standard trick" to at least somewhat ensure atomicity of access to global (shared) variables:

Instead of using

Code: [Select]

SomeGlobal=SomeGlobal+1I wrap that into a function string call, like so:

Code: [Select]

call{ SomeGlobal=SomeGlobal+1; }and call it a day.