Coordinate Systems and Datum Transformations

As this user community knows, collecting high accuracy data requires a working knowledge of coordinate systems, or spatial references, and datum transformations.  The intent of this (long overdue) post is not to deep dive into these topics but rather to summarize how and where datum transformations are used in Trimble Positions.  Let’s look at the two basic workflows.

[Disclaimer: The reader is assumed to have pre-existing knowledge of coordinate systems, datums, datum transformations, and which of each applies to their data.]

ArcPad Workflow

In an ArcPad workflow, coordinate systems come into play in 3 places: at the geodatabase feature class level where the data is stored, at the ArcMap map document (data frame) level where the data is symbolized and “presented”, and in ArcPad where data is collected.  Esri provides form controls for selecting the coordinate system (geographic or projected) for a feature class and also for the data frame of a map document in ArcMap and ArcCatalog.  When the datums differ, you are prompted for a datum transformation in ArcMap.  In ArcPad, there is no functionality to change the coordinate system of a project or layer, but you CAN change the datum of the GNSS (labeled as ‘GPS’ in this case).  Although both autonomous (uncorrected) GNSS and SBAS-corrected GNSS use the global WGS84 datum, many other real-time correction sources (e.g., VRS or single-base) use a regional datum such as NAD83 (CORS96).  In these cases, you would generally change the GNSS datum in ArcPad to reflect the datum of the real-time source.  ArcPad will then handle the transformation locally to ensure that collected data lines up with checked-out data.

Trimble Positions needs to know about this change in GNSS datum in order for further processing.  This is the purpose of the “Coordinate system for real-time corrections (if applicable)” entry box on both the Check In ArcPad AXF files form (and the Import ArcPad QuickProject form).  You should select a geographic coordinate system for the datum used by your real-time correction source in the field.  You will also be prompted for a datum transformation (to WGS84) if one is not already found in the map document.

ArcPad Check In Form

This ensures that all Trimble Positions session and geometry data (extracted from the SSF) is properly stored as WGS84 data in the Trimble Positions office database.  During check-in, ArcMap and the ArcPad data manager components take care of any other transformations necessary between the collected data in the AXF and the underlying feature classes.

During post processing and geometry reconstruction, all internal calculations are performed in WGS84.  However, an additional transformation may be necessary if your map document (data frame) is using a coordinate system with a different datum.  Trimble Positions will look first in the map document to see if a suitable transformation exists (from WGS84).  If your map document contains at least one WGS84 layer in addition to your other non-WGS84 data layers, then this transformation will likely already have been specified by you and picked up by Trimble Positions.  But if your map document contains no WGS84 layers, then you must use the Datum Transformations area of the Administration Window to specify the appropriate datum transformation from WGS84 to your selected coordinate system (datum).  Click the “Set…” button to choose an appropriate geographic coordinate system for your datum.  You will also be prompted for a datum transformation (to WGS84).

Datum Transformation EntryDatum Transformations List

ArcGIS for Windows Mobile Workflow

The ArcGIS for Windows Mobile workflow is similar in that coordinate systems also come into play in 3 places: at the geodatabase feature class level where the data is stored, at the ArcMap map document (data frame) level where the data is symbolized and “presented”, and in ArcGIS for Windows Mobile where the data is collected.  Mobile map caches generated from ArcMap will use the coordinate system and datum of the map document.  If the map document is published to a mobile map service on ArcGIS for Server, then the map service takes the place of ArcMap and uses the coordinate system and datum of the map document.  Esri provides form controls for selecting the coordinate system (geographic or projected) for a feature class and also for the data frame of a map document in ArcMap and ArcCatalog.  When the datums differ, you are prompted for a datum transformation in ArcMap.

When you create a mobile project using Mobile Project Center, the coordinate system of the first added layer (operational or background; from mobile cache or mobile service) is used for the project; all subsequently added layers must use the same coordinate system.  This is viewable by expanding the “more information” section at the bottom of the Project Properties window.

Project Properties - more informationOnce in the field and using GNSS, ArcGIS for Windows Mobile assumes that GNSS data is always WGS84 and will take care of any transformations between WGS84 and the coordinate system of the project.  The Trimble Positions mobile extension will always store session and geometry information in the WGS84 coordinate system.  When working with non-WGS84 real-time correction sources (e.g., VRS or single-base) through a field configuration, it is important to specify the datum and appropriate datum transformation in the settings.

Field Configuration Settings

This ensures that appropriate transformations are applied when storing the data internally in the Trimble Positions session (and geometry constructions) and when returning GNSS positions to ArcGIS for Windows Mobile.

During synchronization of the mobile cache (or mobile service if using ArcGIS for Server), Esri components will handle any transformations necessary between the coordinate system of the project and that of the underlying feature class.  During the GNSS Enabling step earlier in the workflow, you’ll remember that a TrimbleSessions layer was added to the map document.  This layer will always be in WGS84 and so Trimble Positions will always be able to find an appropriate transformation in the map document (if necessary) during post processing and subsequent feature updating.  Thus it is not necessary to use the Datum Transformations area of the Administration Window in the ArcGIS for Windows Mobile workflow.

Perhaps in a future blog post we can look at coordinate systems, projections, and datum transformations in greater detail.  There is quite a bit of information out there already, including some great content from our dealers.  It is a complex topic to be sure.

Posted in Configuration, How To, Under the Hood, Workflow | Leave a comment

Problems Connecting to an NTRIP Source in Desktop

Some of you may have experienced problems trying to create field configurations for NTRIP-based sources (e.g., VRS or single base). Specifically, you receive an ‘Error retrieving the list of sources‘ message a short time after clicking the ellipses button next to the ‘NTRIP Source’ entry box and no mount points will be displayed in the list.

The underlying problem has to do with how the standard .NET methods in Positions Desktop try to parse the headers in the NTRIP response.  Since the response may not include true HTTP headers, it tries to parse the NTRIP headers instead as though they were HTTP headers and therein lies the problem. Ironically, the presence of a proxy server in your network environment resolves the issue since it injects real HTTP headers in the response.

We’re working to resolve this in the next release, but there is a workaround that can be employed now. Locate the ArcMap.exe.config file (e.g., C:\Program Files (x86)\ArcGIS\Desktop10.1\bin) and edit it to add a <system.net> section at the bottom (after the closing </runtime> tag and before the closing </configuration> tag):

  </runtime>
  <system.net>
    <settings>
      <httpWebRequest useUnsafeHeaderParsing="true" />
    </settings>
  </system.net>
</configuration>

For those of you so inclined, there are numerous articles out there about this general .NET option.

Posted in How To, Known Issues, Tips and Tricks | Leave a comment

Problems Downloading CORS Base Station Data – RESOLVED

As of Friday (April 11) afternoon, all issues should be resolved.

Posted in Known Issues, Post Processing | Leave a comment

Problems Downloading CORS Base Station Data

Earlier today we became aware of a problem downloading CORS base station data from the NGS/NOAA servers through Trimble Positions Desktop add-in.  At the beginning of post processing, you will see error messages for the failed downloads.  We are currently investigating the issue but it appears to affect Pathfinder Office and Field Inspector in addition to Trimble Positions.  One workaround would be to note the URLs for the attempted downloads and then download the files manually through a browser.  You can then setup a new post processing profile for those specific files.

We’ll share additional information as we have it.

UPDATE: As of Friday (April 11) afternoon, all issues should be resolved.

Posted in Known Issues, Post Processing | Leave a comment

Support for ArcGIS for Desktop 10.2.1

By now you’ve probably seen the announcements from Esri about the availability of ArcGIS 10.2.1. Although the “major.minor.revision” numbering has been used for a few years in some of the mobile and runtime products, it’s somewhat new to the core ArcGIS releases. In the past, it’s usually been service packs between the “minor” releases of Desktop, Server, and Engine (even though at the DLL level, the service pack number was typically reflected in the “revision” position of the version string). Moving forward, these intermediate releases will likely follow this new convention. One big advantage is that they come with a full installer so that you can install the latest version directly and not have to do the main release and then apply the service pack. You can also upgrade an installation in-place.

Given these new options, there are a few different paths you can take to get to 10.2.1. Although the Trimble Positions Desktop add-in is supported with ArcGIS for Desktop 10.2.1, there are a few things to consider depending on your upgrade path. Here are several of the most common scenarios and the steps you will need to take to get everything working:

  1. You’re running ArcGIS for Desktop 10.1, ArcPad 10.2, ArcGIS for Windows Mobile 10.2, and Trimble Positions Desktop add-in 10.2 and you do an in-place upgrade to ArcGIS for Desktop 10.2.1.
    Trimble Positions Desktop add-in will no longer be registered with ArcMap. You’ll need to perform 3 manual steps: edit the ArcMap.exe.config as described in the release notes (can also be accomplished by uninstalling and re-installing), double-click the PositionsDesktopAddIn.esriAddIn file in “C:\Program Files (x86)\Trimble\Trimble Positions Desktop add-In\AddIns”, and once back in ArcMap, enable the “Trimble Positions Desktop” extension under the Customize…Extensions menu.
  2. You’re running ArcGIS for Desktop 10.1, ArcPad 10.2, and ArcGIS for Windows Mobile 10.2 and you do an in-place upgrade to ArcGIS for Desktop 10.2.1 before installing Trimble Positions Desktop add-in 10.2.
    Trimble Positions Desktop add-in may not be properly registered with ArcMap. If the toolbar is not available, you will need to perform 2 manual steps: double-click the PositionsDesktopAddIn.esriAddIn file in “C:\Program Files (x86)\Trimble\Trimble Positions Desktop add-In\AddIns”, and once back in ArcMap, enable the “Trimble Positions Desktop” extension under the Customize…Extensions menu.
  3. You’re running ArcGIS for Desktop 10.2, ArcPad 10.2, ArcGIS for Windows Mobile 10.2, and Trimble Positions Desktop add-in 10.2 and you do an in-place upgrade to ArcGIS for Desktop 10.2.1.
    Trimble Positions Desktop add-in should remain properly registered with ArcMap. You will only have to do the manual step of editing the ArcMap.exe.config as described in the release notes (can also be accomplished by uninstalling and re-installing).
  4. You do a fresh installation of ArcGIS for Desktop 10.2.1, ArcPad 10.2, and ArcGIS for Windows Mobile 10.2 and then install Trimble Positions Desktop add-in 10.2.
    Trimble Positions Desktop add-in may not be properly registered with ArcMap. If the toolbar is not available, you will need to perform 2 manual steps: double-click the PositionsDesktopAddIn.esriAddIn file in “C:\Program Files (x86)\Trimble\Trimble Positions Desktop add-In\AddIns”, and once back in ArcMap, enable the “Trimble Positions Desktop” extension under the Customize…Extensions menu.

Hope that helps.

Posted in Configuration, How To | Leave a comment

Trimble Positions 10.2

The Trimble Positions Development Team is pleased to announce the release of Trimble Positions 10.2.0.1 for all products. As the name implies, this release supports ArcGIS for Desktop 10.2, ArcGIS for Windows Mobile 10.2, and ArcPad 10.2, in addition to supporting Trimble’s latest GNSS handheld, the Geo 7 Series.

This release addresses usability, bug fixes, and a handful of noteworthy new features:

  • Real-time only workflow. For Positions Mobile extension users, this release introduces the concept of ‘workflow type’ and you can select between Real-Time Corrections + Postprocessing (previously the only option) and Real-Time Corrections Only. In this new workflow type, sessions are not created, metadata is populated in the field, and some out-of-box functionality in the mobile application (copy feature, new feature at same location) is restored.
  • Field-to-office session notes. Positions Mobile extension (and Positions Toolkit) users can now enter descriptive text about the session and this text will be visible to the Positions Desktop add-in user. The columns visible in the sessions grid of the dockable window have been adjusted to display start time, end time, and these notes. Sessions notes can be created and edited for the current session and also for previously saved (unsynchronized) sessions by clicking the appropriate entry in the New Sessions in Cache list in the GNSS Sessions task.
  • Nested point feature collection. In this version of the Positions Mobile extension (and Positions Toolkit), you can pause streamed line collection to collect nested point features. This feature is available as a menu option in the Form View geometry page. You can select from any of the point layers in the current project.
  • List of satellites used. Clicking on the Satellites: text on the GNSS Status page of the Positions Mobile extension will expand the area to show a list of satellites used in the solution. Satellites are listed by constellation.
  • Legacy workflow support. This version adds basic support for a few legacy workflows that have been requested: create data dictionary (DDF) from the layers in the current map document (for the TerraSync™ software), and create a projection file (PRJ) from the coordinate system used in the current map document (for the GPS Pathfinder® Office software).

The release notes contain the full listing of new functionality and bug fixes and should be consulted for further detail. Check back here in the next couple of weeks for demonstration videos highlighting some of the new functionality in the context of different workflows.

Before we go, an important note: This release includes an upgrade to the schema of the Trimble Positions Desktop database. As a reminder, this is the Jet/MDB or ODBC data source that you configured during installation using the Trimble Positions Desktop Configuration application available in the Start Menu. After installing Trimble Positions Desktop add-in 10.2.0.1, run the Configuration application, and click the ‘Test current configuration‘ link. This will alert you that the schema needs to be upgraded and that all existing data (projects, processing profiles, field configurations, etc.) will be preserved. Click the ‘Yes‘ button, wait a few seconds, and then re-click the ‘Test current configuration‘ link to verify that the schema was upgraded.

UpgradeSchema

Posted in What's New | Leave a comment

Positions ArcPad Extension for Windows – Installation Problems

In August of this year, we released the long-awaited Positions ArcPad extension for Windows.  The 10.0.0.2 installation package includes installers for both Windows and Windows Mobile.  For those of you who have installed the Positions ArcPad extension on Windows, you likely have come across some errors either during installation or when trying to run ArcPad.  Unfortunately, the installer for Windows is missing a required Microsoft Visual C++ runtime redistributable.

To rectify this:

  1. Download the Visual C++ 2008 SP1 Redistributable Package ATL Security Update from Microsoft: http://www.microsoft.com/en-us/download/details.aspx?id=11895. As ArcPad is a 32-bit application, you will only need the x86 version (vcredist_x86.exe).
  2. Uninstall Positions ArcPad extension.
  3. Uninstall ArcPad.
  4. Install the runtime redistributable.
  5. Re-install ArcPad.
  6. Re-install Positions ArcPad extension.

That should resolve the problem.  We’ve heard reports of it working without steps 3 and 5 but included them here regardless.

Posted in Known Issues | Leave a comment

US Government Shutdown & CORS Data

Due to the US government shutdown all NGS/CORS services are unavailable (UFCORS, CORS ftp). This means that data cannot be post-processed against CORS base stations or services unless the data has previously been downloaded. If this affects you, please contact your Trimble dealer as they may be able to suggest other sources of base station data for your area.

Posted in Known Issues | Leave a comment

Setting Up a Shared Trimble Positions Desktop Database on SQL Server

The goal is to setup a shared SQL Server database called TPDatabase1 on SQLServer1 that will allow User1 on PC1 and User2 on PC2 to share their Trimble Positions projects, post-processing profiles, and field configurations.  This will be done by using Windows Authentication.  Let’s assume that User1 and User2 are Windows domain users and at least User1 is a member of the administrators group for the SQL Server installation on SQLServer1.  Also, User1 has SQL Server Management Studio (SSMS) installed on either PC1 or SQLServer1 so that he/she can connect to SQLServer1.  Finally, PC1 and PC2 have installed the SQL Server Native Client (at or above the SQL Server version), Esri ArcGIS for Desktop, and the Trimble Positions Desktop (TPD) add-in.

  1. Launch SSMS on PC1 or SQLServer1, connect to SQLServer1 as User1, find the ‘Databases’ node in the Object Explorer panel and right-click to create a ‘New Database’.
  2. In the SSMS dialog that opens, specify the name (TPDatabase1) and click ‘OK’.
  3. In the SSMS Object Explorer panel, find the ‘Logins’ node within the ‘Security’ node and right-click to create a ‘New login’.
  4. In the SSMS dialog that opens, specify the login name (User2).
  5. In the same SSMS dialog, click the ‘User Mapping’ link at left and note the list of databases displayed.  Click the checkbox next to TPDatabase1 and when the ‘Database role membership’ area is enabled below, click the checkbox next to ‘db_owner’. [Note: the combination of ‘db_datareader’ and ‘db_datawriter’ would likely be sufficient but this has not been explicitly tested.]  Click ‘OK’ to complete this.  SSMS can now be closed.
  6. Launch the TPD Configuration tool on PC1 and select the option to ‘Add/Edit a new configuration’.
  7. In the TPD Configuration screen, select ‘Single user’ or ‘All users on this PC’ as appropriate.
  8. In the same screen, select the radio button for ‘I will setup a database connection myself (Advanced)’.
  9. In the same screen, select the option to ‘Manage ODBC data sources’.
  10. In the ODBC Data Source Administrator window that opens, select the ‘System DSN’ tab.
  11. From the ‘System DSN’ tab, click ‘Add…’ and select an appropriate driver from the list (e.g., SQL Server Native Client 10.0), and click ‘Finish’.
  12. In the ‘Create a New Data Source’ (DSN) wizard, provide a name for the data source (we’ll call it TPonSQL), specify the server instance name (SQLServer1), and click ‘Next’.
  13. In the authentication page of the DSN wizard, leave the radio button for ‘With Integrated Windows authentication’ checked and click ‘Next’.
  14. In the next page of the DSN wizard, click the checkbox next to ‘Change the default database to’, select TPDatabase1 from the list, and click ‘Next’.
  15. In the final page of the DSN wizard, click ‘Finish’.  In the confirmation dialog that opens, click ‘Test Data Source…’ to confirm a proper connection.  Click ‘OK’ twice to get back to the ODBC Data Source Administrator screen.
  16. From the ODBC Data Source Administrator screen, click ‘OK’ to get back to the TPD Configuration screen.
  17. In the TPD Configuration screen, open the ‘ODBC data source’ select list and choose the TPonSQL entry that you just created.
  18. In the same screen, click the ‘Test data source’ link and at the prompt to create schema, click ‘Yes’.  Wait for the confirmation dialog and click ‘OK’.
  19. In the same screen, click the ‘Apply’ button.
  20. In the original TPD Configuration screen, click ‘Exit’.  This completes the configuration of PC1.
  21. As User2 logged in on PC2, repeat steps 6 through 20 and note that in step 18, you will not be prompted to create schema as the schema already exists and you have access to it.

Notes:

  • The SQL Server instance name has been simplified for this example.  It may take on the form SERVER\INSTANCE,PORT.
  • The usernames have also been simplified for this example.  In a domain (Active Directory) environment, they would take on the form DOMAIN\USER.
  • This example makes use of ‘System DSN’ entries which are specific to the machine.  For Citrix-type environments where users may login to different physical computers, the use of ‘User DSN’ entries should be considered in conjunction with the ‘Single user’ configuration type.
  • This procedure should work equally well with SQL Server Express provided the user also downloads and installs SQL Server Management Studio Express.
  • Although this example was given for SQL Server, it could also be done with Oracle, PostgreSQL, etc. assuming the user has the requisite drivers and management tools installed.
  • If SQL Server has been installed with ‘Mixed Mode Authentication’, it may be easier to create a SQL login with full access to the TPD database and then use this SQL login in the DSN entries on both computers.  SQL Server security best practices (and perhaps your IT department) should be consulted.
Posted in Configuration, How To, Tips and Tricks | Leave a comment

Trimble Positions ArcPad Extension Now Supports Windows!

We’re excited to announce that the Trimble Positions ArcPad extension now includes support for Esri ArcPad on Windows.  This provides GIS customers looking to couple ArcPad with powerful tablet PCs, such as the Trimble Yuma 2, with the ability to post-process and analyze field data collected on a tablet.  Available for download now, the installation package for the Trimble Positions ArcPad extension v10.0.0.2 includes installers for both Windows and Windows Mobile.  The Windows version has the same language support that the Windows Mobile version does.

Look for the new “Trimble Positions Software Suite – August 2013 (current)” section of the Technical Support – Downloads area at www.trimble.com/positions.  Note that all version numbers remain the same as the previous release as there have been no other changes in the software; all other installation packages are identical to the previous release.  Esri pre-requisites and version support also remain the same.

Posted in What's New | Leave a comment