View source for Howto:Using FlightGear as an Image Generator (IG)

<!--{{Stub}}-->
[[File:FAA 7996e.jpg|thumb|NIEC's A320 simulator with in house Image Generator]]

{{Rendering}}

== Motivation ==
Replacing our in-house Image Generator with FlightGear's visuals.  There are two major features which are motivating the switch to FlightGear.  
# the scenery detail and the tools to update it as needed it lacking in our current system.  
# the interpretation of weather information into weather visuals is outstanding.

== Background ==
FlightGear is a highly configurable flight simulator, where many subsystems can be customized or even disabled entirely. 
This includes the capability to disable the built-in flight dynamics engine (FDM) and use an external one (the so called null fdm), this makes it possible to also link one fgfs instance slaved to another master instance, where the slave instance would use the positional information from the master instance.
 
In addition, it is possible to feed in control inputs from external hardware/software (middleware).

Furthermore, FlightGear supports a number of multi-screen features, namely the capability to set up multiple computers in a master/slave fashion, where slaved instances use data from the master instance, as well as a sophisticated XML-configurable multiscreen mode for individual instances with multiple graphics cards and screens.

This makes it possible to use FlightGear in conjunction with 3rd party flight dynamics/simulators, where FlightGear would primarily serve as an  image generator, with the flight dynamics and control inputs provided by external means.

It would be a good idea to start by going through related articles on the wiki, e.g. to name just a few <ref> {{cite web
  | url    = http://forum.flightgear.org/viewtopic.php?p=282283#p282283
  | title  = <nowiki>Re: Hardware recommendations for multi-screen sim</nowiki>
  | author = <nowiki>Hooray</nowiki>
  | date   = Apr 12th, 2016
  | added   = Apr 12th, 2016
  | script_version = 0.25
  }}
</ref>:
* [[Hardware recommendations]]
* [[FlightGear Expo Checklist]]
* [[Presentation Recipe]]

== Objective ==
Document various interfacing mechanisms that are suitable for using FlightGear as an '''IG''' (Image Generator) with the built-in FDM disabled, and in conjunction with a 3rd party flight simulator/FDM (flight dynamics model).


Immediate goals are:
* get aircraft data into flightgear.  Starting with a simple airport simulation at KPHL with lots of departures and arrivals. (probably via the fgms multiplayer server/[[Multiplayer protocol]]) {{Progressbar|10}}
* use master/slave setup to drive several of our displays (as per {{readme file|IO}}) [[Howto:Multi-computing FlightGear]] {{Progressbar|10}}
* locally add our models to the Flightgear scenedata.  Which FG piece would help in this? (the model manager at runtime, which is accessible via properties under /model or scenery xml files read by the tile manager) {{Progressbar|10}}
* get elevation data from the scenery data, either in our current format or dynamically in a server type setup.

== Potential Solutions ==
* to add static 3D models to the FlightGear scene, you would want to use the so called "model manager" - this can be done via properties (see <code>/models</code> in the property tree), so that you can access the model manager via Nasal, but also using one of the built-in IPC mechanisms, e.g. [[Telnet usage|telnet/props]].
*  if you need a better/faster method, you would want probably want to look at the underlying C++ code, and either extend/customie that, or re-implement the whole thing on top of a binary protocol that runs in a separate thread
*  likewise, dynamical elements can be added using the AI subsystem (again, via properties: /ai) and/or via the multiplayer subsystem - the latter is probably easier to use if you already have external data, because you can use the multiplayer protocol and some middleware to inject traffic into fgfs - under the hood, the multiplayer system is also using the ai subsystem
*  master/slave synchronization may become an issue - so that it would be easier to really focus on the multiplayer component, and let all instances use the same local fgms server, so that you don't have to reinvent things from scratch.
*  note that the fgms multiplayer server is pretty rudimentary and it too flexible, so for lots of traffic it will require quite a bit of bandwidth, so that a LAN may be required for reasonable latencies/performance
*  equally, if are not interested in hacking any of the underlying C++ code, you could use the scenery/terrasync infrastructure and its support for placing 3D objects to store your models all on a network accessible drive/folder, and then set up each fgfs slave instance to use the same NFS/samba mounts. 


That's basically the "lazy way", i.e. you don't need to modify tons of C++ code to get your setup going using NFS shares and the multiplayer system, so that all your stated requirements would be fulfilled directly - however, it's also the one least likely to benefit FlightGear as a whole, because you are really just using existing features in a creative way to make up for missing features that would be required for your use-case - that will particularly show when it comes to synchronizing/replicating runtime state that isn't easily replicate - such as e.g. having weather (winds, clouds, precipation etc) in sync between all master/slave instances.<ref>{{cite web
  | url    = http://forum.flightgear.org/viewtopic.php?p=276543#p276543
  | title  = <nowiki>Re: External targets and Multiple Screens</nowiki>
  | author = <nowiki>Hooray</nowiki>
  | date   = Feb 18th, 2016
  | added   = Feb 18th, 2016
  | script_version = 0.25
  }}
</ref>

== Status ==
Last updated: 03/2016
=== Phase 1 ===
* Simgear and Flightgear are compiled and installed from the correct git repos found here: https://sourceforge.net/p/flightgear/_list/git?source=navbar
* Added flightgear-fgdata to our data repo so we don't kill your servers with 10's of screens.
* Added the [[A320neo]] aircraft to our repo.  The A320 will be used ad the default aircraft.
* http://www.inkdrop.net/dave/multimon.pdf multiscreen tested on a master with a single slave.  works very nicely.
* Using ~/.fgfsrc files for master and slave settings. Will post files and settings when we have them working.
*  Installed on ten machines and configured.  FOV=38.5 and heading angles are 22.5 deg each.  Using .fgfsrc on each computer for the display configuration data and an nfs drive for the scripts and code.  the script has all the common fgfs arguments(details needed)
* Middleware for cockpit is finished and appears to be mostly working.  We are using multicast to broadcast position to all of the slaves (details need).  There is an issue where the aircraft position is over water in our cockpit, but running on our desktops everything works...  yeah... 
* Import of FLT models have unshade or black rendering on the side which is opposite the sun. The side should not be that dark! But the models are being put in so that is the plus.
* Hacked in the rain.  It looks to me like there is no reason to reset the scenery if it's already been initialized.  I see it is called twice, however a stacktrace reveals it is called from the same exact line... twice... what?
* Found the -DCMAKE_INSTALL_PREFIX=$PWD/install flag to make jumping around from version to version easier.  Now if there was a way to do the scenery more easily.  3.7.0 is the oldest in the sf git repo.
* FlightGear is now working on all nine computers and tested well with our middleware

=== Phase 2 ===
* Send Storm cloud data to all nine computers and evaluate the rendering across the screens:  Need good data, need a way to send it to all nine displays.  

Using techniques from the Local Weather project we discovered we could add clouds as "models" which will render over the IG's consistently.  Looking at using java/python/perl telnet interface to control weather.

* Add "external" aircraft to the simulation using multiplayer packets.

=== Gallery ===
<gallery>
</gallery>

== Hardware ==
{{Stub}}

== Roadmap ==
<!--
{{Cleanup|this would be a good section to test the new repo link templates, i.e. transclusion of nested templates}}
-->
=== Prerequisites ===
Using Fedora 21
Yum install all dependencies

<syntaxhighlight lang="bash">
sudo yum install gcc-c++ cmake openal-devel glut-devel libXmu-devel libXi-devel qt-creator openal-devel plib-devel OpenSceneGraph-devel zlib-devel boost-devel bzip2-devel fltk-fluid fltk-devel
</syntaxhighlight>

=== Building SimGear ===
Download and build simgear:

<syntaxhighlight lang="bash">
git clone git://git.code.sf.net/p/flightgear/simgear
cd simgear
mkdir build
cd build
cmake-DCMAKE_INSTALL_PREFIX=../../fglatest  ..
make
sudo make install
</syntaxhighlight>

=== Building FlightGear ===
Download and build FlightGear:

<syntaxhighlight lang="bash">
git clone git://git.code.sf.net/p/flightgear/flightgear
cd flightgear
mkdir build
cd build
SIMGEAR_DIR=../../fglatest/ cmake -DSIMGEAR_INCLUDE_DIR=../../fglatest/include/ -DCMAKE_INSTALL_PREFIX=../../fglatest ..
make
sudo make install
</syntaxhighlight>

=== Getting the Base Package ===
Get the fgdata directory:

{{Fgdata clone}}

=== Scenery ===
Get additional scenery using [http://open744.github.io/terramaster/ TerraMaster]
<syntaxhighlight lang="bash">
java -jar terramaster.jar
</syntaxhighlight>

=== Installation ===
Installing on the host and nine image generators:
Compiled for Fedora 21 as per the above instructions.  tar'ed the fglatest directory and put it on an NFS shared directory named /data/fg_ig.

Each of the nine IG use autofs to mount the /data/fg_ig directory.  
<syntaxhighlight lang="bash">
ln -s /net/fghost/data/fg_ig /data/fg_ig
</syntaxhighlight>

There is a runfg.sh script and an fgfs_cmdline_option file in the shared fg_ig directory.

runfg.sh contains command to turn off screen saver and use the fgfs_cmdline_option file as command line options:
<syntaxhighlight lang="bash">
export DISPLAY=:0 
xset s noblank
xset s noexpose
xset s off
xset -dpms

/data/fg/fglatest/bin/fgfs --config=rcfs-config.xml $(< fgfs_cmdline_option) $@
</syntaxhighlight>

fgfs_cmdline_option contains options common to all the IGs:
<syntaxhighlight lang="bash">
--prop:/sim/ati-viewport-hack=false

--fg-root=/data/fg_ig/fgdata

--enable-fullscreen

--aircraft=A320neo
--disable-sound
--httpd=5400

--fdm=external
--native-fdm=socket,in,40,225.225.55.15,5515,udp
--native-ctrls=socket,in,40,225.225.55.16,5516,udp


--timeofday=noon
--airport=kacy

--enable-clouds
--enable-clouds3d

--prop:/sim/rendering/shaders/skydome=true
--prop:/sim/rendering/random-vegetation=2.5
--prop:/sim/rendering/shaders/quality-level=-1
--prop:/sim/rendering/shaders/contrails=1
--prop:/sim/rendering/shaders/crop=1
--prop:/sim/rendering/shaders/custom-settings=true
--prop:/sim/rendering/shaders/forest=1
--prop:/sim/rendering/shaders/generic=1
--prop:/sim/rendering/shaders/landmass=6
--prop:/sim/rendering/shaders/lights=4
--prop:/sim/rendering/shaders/model=3
--prop:/sim/rendering/shaders/quality-level-internal=5
--prop:/sim/rendering/shaders/transition=6
--prop:/sim/rendering/shaders/urban=1
--prop:/sim/rendering/shaders/water=5
--prop:/sim/rendering/shaders/wind-effects=2

--disable-ai-traffic

--enable-horizon-effect
--enable-enhanced-lighting
--enable-distance-attenuation
--enable-specular-highlight

--prop:/sim/rendering/multi-sample-buffers=true
--prop:/sim/rendering/multi-samples=4

</syntaxhighlight>

=== IG Setup ===
Each Image Generator is a slave that uses its own [[Fgfsrc|.fgfsrc]] file which off-set the headings by 22.5 degrees as shown in this diagram.

<gallery>
File:OTW-Display-Layout.png |Display layout
</gallery>

Here is a sample of Cockpit Image Generator 01 (CPIG01), which is the screen on the far left, .fgfsrc file.
<syntaxhighlight  lang="bash">
# $HOME/.fgfsrc

--prop:/sim/view[0]/config/field-of-view=38.5
--prop:/sim/view[0]/config/heading-offset-deg=90

--prop:/sim/view[0]/config/x-offset-m=0
--prop:/sim/view[0]/config/pitch-offset-deg=0
--prop:/sim/view[0]/config/default-field-of-view-deg=38.5

</syntaxhighlight>

===Aircraft Model===
The front of the A320 model was removed with the Blender.  We did this because the view would shift if we turned off the model.  The accuracy of the A320neo model was perfect.  The control panel, joystick, foot pedels, even the wind shield matched up with our shell.  Because of the detail we assume the view height and forward position are also very accurate.

===Weather===
Using techniques from the Local Weather project we discovered we could add clouds as "models" which will render over the IG's consistently.

== Approaches ==
There are a lot of different way to skin this particular cat with FG :).
At first glance, for drawing OTW I'd suggest using the IO option.  
I'd suggest reading through the following files in the Docs/ directory:
* {{readme file|IO}}
* {{readme file|protocol}}
* {{readme file|multiscreen}}
The last is particularly handy as it shows how to set up view fustrums for each of the OTW views you have.
Finally, as well as getting help from this forum, you might also want to contact the -devel mailing list.  There are some people there who have done work in this area who don't frequent this forum.<ref>{{cite web
  | url    = http://forum.flightgear.org/viewtopic.php?p=276114#p276114
  | title  = <nowiki>Re: External targets and Multiple Screens</nowiki>
  | author = <nowiki>stuart</nowiki>
  | date   = Feb 14th, 2016
  }}
</ref>

=== Slaving ===
With README.IO and README.protocol (in $FG_ROOT/Docs) documenting ways to interlink/synchronie multiple fgfs sessions (instances) that will usually reside on different computers. README.multiscreen is primarily aimed at the single-instance/multi-window/multi-screen use-case. However, these operating modes are not mutually-exclusive necessarily.
The master/slave method is also explained at: [[Property Tree/Native Protocol Slaving]]

I/O sockets are documented at: [[Property Tree/Sockets]]
<ref> {{cite web
  | url    = http://forum.flightgear.org/viewtopic.php?p=276118#p276118
  | title  = <nowiki>Re: External targets and Multiple Screens</nowiki>
  | author = <nowiki>Hooray</nowiki>
  | date   = Feb 14th, 2016
  }}</ref>

* http://wiki.flightgear.org/Property_Tree/Native_Protocol_Slaving
* http://wiki.flightgear.org/Howto:Multi-computing_FlightGear

=== Custom Protocols ===
The other option being a custom coded network protocol with a focus on your specific requirements, this is commonly done for things like proprietary/3rd party software, refer to $FG_SRC/Network for examples, e.g.: https://sourceforge.net/p/flightgear/flightgear/ci/next/tree/src/Network/
<ref> {{cite web
  | url    = http://forum.flightgear.org/viewtopic.php?p=276118#p276118
  | title  = <nowiki>Re: External targets and Multiple Screens</nowiki>
  | author = <nowiki>Hooray</nowiki>
  | date   = Feb 14th, 2016
  }}
</ref>

=== Generic Protocol ===
A "hybrid" mechanism is the so called "generic protocol" which is basically a simple property-synchroniation protocol that is XML configurable (but not scriptable).
This will not necessarily be useful in isolation, but it can be used to "augment" the other mechanisms - which will be particularly important when dealing with subsystems that are currently not aware of the distributed/multi-instance use-case (think multiplayer, fsweekend/linuxtag, image generator setups etc)
The corresponding XML files (PropertyList XML) are to be found in $FG_ROOT/Protocol: [[Generic protocol]]
<ref> {{cite web
  | url    = http://forum.flightgear.org/viewtopic.php?p=276118#p276118
  | title  = <nowiki>Re: External targets and Multiple Screens</nowiki>
  | author = <nowiki>Hooray</nowiki>
  | date   = Feb 14th, 2016
  }}
</ref>

=== HLA, DIS and CIGI ===
There are several more or less complete attempts at bringing DIS, HLA and CIGI to FlightGear (in that order), some of these have not yet been communicated on the devel list, but you can find the corresponding patches/forks online.<ref> {{cite web
  | url    = http://forum.flightgear.org/viewtopic.php?p=276118#p276118
  | title  = <nowiki>Re: External targets and Multiple Screens</nowiki>
  | author = <nowiki>Hooray</nowiki>
  | date   = Feb 14th, 2016
  }}
</ref>

<references/>

== Challenges ==
90% of the subsystems in FlightGear are not developed/maintained with the distributed/multi-instance use-case in mind, which shows when using features like AI traffic, weather engines or multi-pilot aircraft.
<ref> {{cite web
  | url    = http://forum.flightgear.org/viewtopic.php?p=276118#p276118
  | title  = <nowiki>Re: External targets and Multiple Screens</nowiki>
  | author = <nowiki>Hooray</nowiki>
  | date   = Feb 14th, 2016
  }}
</ref>

The major issue you should be aware of here is that all the slaving/multi-screen stuff is basically pre-dating the ongoing multicore/HLA work by many years, so there isn't any kind of holistic scheme in place, but just a variety of different (and often incompatible) features and mechanisms that you would need to review, to determine the strengths and weaknesses of each approach - with your particular requirements in mind.
The corresponding restrictions are most likely to show up in multi-instance setups showing the same cockpit (especially anything involving MFDs/glass cockpits), but also environmental state (weather), and anything involving AI objects (think AI traffic).
Depending on your requirements, these may require a little more attention or not - i.e. even if only by disabling such features.<ref> {{cite web
  | url    = http://forum.flightgear.org/viewtopic.php?p=276168#p276168
  | title  = <nowiki>Re: External targets and Multiple Screens</nowiki>
  | author = <nowiki>Hooray</nowiki>
  | date   = Feb 15th, 2016
  }}
</ref>

Another thing worth taking into consideration that you may need a mechanism to exchange/patch certain simulator-side data with your external FDM and vice versa - unless the FDM and the scenery engine are using the same underlying terrain database - e.g. imagine differences in altitude above ground level (AGL); you may get away with a applying some kind of dynamical offset for lat/lon and altitude differences between the FDM and the IG (FlightGear), but according to past discussions on the matter (e.g. those wanting to hook up the Aerowinx 747 precision simulator/PSX), this is one of the less-obvious challenges - and it may make sense to establish some kind of middleware to negotiate between the underlying FDM/scenery data.
To some extent, the osgEarth integration was facing similar issues - e.g. it had to patch up internal APIs for getting the current lat/lon/altitude, so that they're using the osgEarth instead of the default scenery engine.
Ideally, FlightGear would provide some kind of dedicated IPC mechanism that could be used for these purposes - with different FDMs and/or flight simulators in mind, where FG would solely be responsible for acting as an IG.
<ref> {{cite web
  | url    = http://forum.flightgear.org/viewtopic.php?p=276665#p276665
  | title  = <nowiki>Re: External targets and Multiple Screens</nowiki>
  | author = <nowiki>Hooray</nowiki>
  | date   = Feb 19th, 2016
  | added   = Feb 19th, 2016
  | script_version = 0.25
  }}
</ref>

* https://www.mail-archive.com/search?q=aerowinx&l=flightgear-devel%40flightgear.org
* https://www.mail-archive.com/search?q=ps1&l=flightgear-devel%40flightgear.org
* https://www.mail-archive.com/search?l=flightgear-devel%40flightgear.org&q=image+%2B+generator&x=0&y=0
* https://www.mail-archive.com/search?l=flightgear-devel%40flightgear.org&q=scenery+%2B+generator&x=0&y=0

=== Multi-instance Weather Synchronization ===
Regarding the [[index.php?title=Howto:Using FlightGear as an Image Generator %28IG%29&amp;curid=15237&amp;diff=96409&amp;oldid=95939|latest wiki edits]], for anything involving weather (and specifically the [[Advanced weather|Advanced Weather system]]), you will probably want to get in touch with Thorsten to determine the limitations of the existing system and feasible workarounds to synchronie/replicate weater across multiple instances in a holistic fashion.
<ref>{{cite web
  | url    = http://forum.flightgear.org/viewtopic.php?p=280803#p280803
  | title  = <nowiki>Synchronizing/replicating weather data across fgfs instances</nowiki>
  | author = <nowiki>Hooray</nowiki>
  | date   = Mar 28th, 2016
  | added   = Mar 28th, 2016
  | script_version = 0.25
  }}
</ref>

=== Canvas Avionics ===
{{Main article|Canvas ND Framework#Encapsulating Properties}}
[[File:NDSourceDriverHash-concept.png|thumb|Screenshot demonstrating how the Canvas-based ND was designed  to use an encapsulated "data souce" abstraction to hide implementation details, so that the same ND code can be used for creating independent ND instances that support AI/MP (or [[Dual Control]]) aircraft for driving the MFD<ref>http://forum.flightgear.org/viewtopic.php?f=4&t=28965&p=280612#p280612</ref>. Note that this was prototyped in late 2013 originally<ref>http://forum.flightgear.org/viewtopic.php?p=193660#p193660</ref>, and may not have received much testing/attention lately.]]

=== ICAO Requirements ===
{{FGCquote
|1= Theme of image generator in flight simulator is close to me too. There is good document from ICAO, describing what must be presented in flight simulator - [http://123.127.67.21/fagui/ICAO9625-AN938(english http://123.127.67.21/fagui/ICAO9625-AN938(english).pdf]

For example, point light must not exceed 5 arc minutes. Flightgear light system currently draws much more bigger lights.
|2= {{cite web
  | url    = http://forum.flightgear.org/viewtopic.php?p=280805#p280805
  | title  = <nowiki>Re: Synchronizing/replicating weather data across fgfs insta</nowiki>
  | author = <nowiki>Soitanen</nowiki>
  | date   = Mar 28th, 2016
  | added   = Mar 28th, 2016
  | script_version = 0.25
  }}
}}

== Related ==
=== Multiple Screens ===
* $FG_ROOT/Docs/README.multiscreen
* http://www.inkdrop.net/dave/multimon.pdf
* [[Howto:Configure camera view windows]]
* [[FGViewer]] (HLA-based standalone viewer)
* [[CompositeViewer Support]]

=== IG Use Case ===
* FSWeekend/LinuxTag presentation recipe
* [[GUI Messages]]
* http://forum.flightgear.org/viewtopic.php?f=6&t=6586&p=56841&hilit=#p56841

{{FGCquote
|1= i'm trying to use flightgear as a scenery generator for PS1. for that purpose i have installed a program called broker (gets info from PS1) and fgvisual (takes data from broker and sends it to flightgear). it looks like fgvisual gets its data from broker ok, and then sends it on to the right IP address and port. but the problem is that flightgear doesn't respond. it just sits there on the runway and refuses to budge. PS1, broker and fgvisual run in a virtual machine under XP, flightgear 2.8 runs under OSX on the same hardware.


|2= {{cite web
  | url    = http://forum.flightgear.org/viewtopic.php?p=171695#p171695
  | title  = <nowiki>flightgear and </nowiki>
  | author = <nowiki>harry747</nowiki>
  | date   = Dec 2nd, 2012
  | added   = Dec 2nd, 2012
  | script_version = 0.25
  }}
}}

=== Slaving ===
* {{readme file|IO}} (master/slave setups)

=== Multiplayer ===
* [[Howto:Multiplayer#Advanced_usage_of_multiplayer]]
* [[FlightGear Multiplayer Server]]
* [[Howto:Set up a multiplayer server]]
* [[Multiplayer protocol]]

=== Misc ===
* [[High-Level Architecture]] (HLA)
* [[FlightGear CIGI Support (Common Image Generator Interface)]]
* {{Issue|924}}
* [[Distributed Interactive Simulation]] (outdated, but still relevant background info)

== References ==
{{Appendix}}