Rick Strahl's FoxPro and Web Connection Web Log
 

Single File Image Uploads with plUpload and Web Connection


No comments
Monday, April 20, 2015, 10:31:00 AM

I get a lot of questions about uploading files in regards to Web Connection. Uploading is a thorny problem especially if you need to upload multiple files or if you want to use asynchronous uploads using AJAX rather than submitting files through standard <input type=”file”> elements which suffer from many problems. Using asynchronous upload components allow getting around the 16 meg string limit in FoxPro and Web Connection and they allow you to do the uploads in the background while you can display progress information and keep your UI otherwise active.

Web Connection has had support for plUpload for some time now. There’s a plUpload server handler that allows you to use the plUpload Queue component which is a visual UI component that’s provided by the plUpload library. Web Connection provides a generic plUploadHandler class that can be used on the server side to capture uploaded files.

The examples show how to do this using the plUpload Queue component which is bulky UI control. Here’s what the full UI that you can drop on any form looks like:

You can check out this example and the following single file upload example on the Web Connection Samples page:

http://west-wind.com/wconnect/webcontrols/plUploadSample.htm

Single File Uploads

The full component is useful for batch uploads, but sometimes you don’t want or need this rather bulky UI to handle file uploads. For example I recently needed to add image uploads to the West Wind Message Board and in order to do this I really wanted a much simpler UI that simply triggers the image upload from a button:

ImageUpload

There’s a simple button that when clicked allows you to pick a single file (although you can also have it pick multiples) and then immediately starts uploading the image to the server. When done the image URL returned is then embedded into the user’s message at the current cursor position.

Let’s see how we can do this using the simpler plUpload base API.

Using the base plUpload API

plUpload includes a number of different upload components but behind it all sits a base uploader API which doesn’t have any UI components associated with it. For my image uploader I don’t want any extraneous UI – I only want to upload the file and provide some basic progress information.

To do this we can use the core plUpload API. Here’s how this works.

Let’s start with the Add Image Dialog HTML which displays the modal dialog above:

<div id="InsertImageDialog" class="dialog" style="display: none; min-width: 320px; width: 80%"> <div class="dialog-header">Insert Image</div> <div class="dialog-content"> <label>Web Image Url: <small>(external source ie.Flickr,Imgur, DropBox etc.)</small></label> <input type="url" id="txtImageLink" style="width: 98%" /> <button type="button" id="btnImageSelection" class="bigbutton" style="margin-top: 5px;">Insert Web Image</button> <div id="container" style="margin-top: 30px;"> <label>Upload an Image:</label> <button id="btnUploadFile" onclick="return false;" class="bigbutton">Select file to Upload</button> </div> </div> </div>

Next we need to add the relevant plUpload script to the page.

<script src="scripts/plUpload/plupload.full.min.js"></script>    

Then we need to configure the plUpload Javascript code which can live in a script tag on the bottom of the page or as is the case here inside of a page level JavaScript file.

var uploader = new plupload.Uploader({ browse_button: 'btnUploadFile', // you can pass in id...
url: "ImageUpload.wwt",




runtimes: 'html5,flash,html4',
container: document.getElementById('container'), // ... or DOM Element itself url: "ImageUpload.wwt", chunk_size: '64kb', // Resize (downsize really) images on clientside if we can resize: { width: 1024, height: 768, quality: 85 }, filters: { max_file_size: '4mb', mime_types: [ { title: "Image files", extensions: "jpg,gif,png" } ] }, // Flash settings flash_swf_url: 'scripts/plupload/js/Moxie.swf', init: { PostInit: function () { }, FilesAdded: function (up, files) { // start uploading - we only accept one file uploader.start(); }, UploadProgress: function (up, file) { showStatus("Upload Progress: " + file.percent + "% complete",3000); }, Error: function (up, err) { showStatus("Upload failed: " + err.code + ": " + err.message); },

FileUploaded: function(up, file, response) { uploader.removeFile(file); var imageUrl = response.response; if (imageUrl) { markupSelection("<<img src=\"" + imageUrl + "\" />>"); $("#InsertImageDialog").modalDialog("hide"); } }, UploadComplete: function(up, files) { } } }); uploader.init();

The plUpload script code is pretty descriptive so not much explanation is needed. The most important propertie here is the browse_button property which is an id pointing at a button or link that when clicked triggers the image upload and the url property which points at the server target URL that will response to the plUpload file chunks that are sent to the server.

The interesting stuff happens in the event handlers.

Since I’m only dealing with a single file selection, I can use the FilesAdded event to immediately start the file upload under program control. This event fires whenever you select one or more files, and if you use a single button it makes sense to just kick off the upload without further user confirmation.

For progress and error information I use the ww.jquery.js showStatus()  function which is a quick and easy way to display status information on a status bar on the bottom of the form.

The most important piece though is the FileUploaded event which is used to actually confirm the file upload and capture the generated filename that the server saved. The function receives the upload component, the individual file object and and HTTP response object. The main thing we’re interested in the response property of the response object which provides a fully qualified image URL that points at the image that the server saved. This value is captured, an <img> tag created and then pasted into the text control at the current cursor position.

Handling the Image plUpload on the Server Side

As mentioned earlier Web Connection includes a plUploadHandler class that makes it pretty straight forward to handle uploads. The class basically works in conjunction with a wwProcess class and handles the plUpload file transfer chunks and puts the files together on the server in small chunks. This makes it possible for example to post files larger than 16 megs to the server as well as the file is sent in small chunks that are progressively appended to a file on the server.

To implement the server side you’ll create two methods:

  • A standard wwProcess EndPoint Method
  • An OnUploadComplete Event that is fired when the upload is complete

The first is the actual endpoint method that is referenced by the plUpload component. If you look back on the JavaScript configuration you see that it points at ImageUpload.wwt which translates to the following method in my wwThreads Process class:

FUNCTION ImageUpload()
 
*** Make sure plUploadHandler is loaded
SET PROCEDURE TO plUploadHandler ADDITIVE 
 
LOCAL loUpload as plUploadHandler
loUpload = CREATEOBJECT("plUploadHandler")
 
*** Upload to temp folder
loUpload.cUploadPath = ADDBS(THIS.oConfig.cHtmlPagePath) + "temp"
IF(!IsDir(loUpload.cUploadPath))
    MD (loUpload.cUploadPath)
ENDIF    
 
BINDEVENT(loUpload,"OnUploadComplete",THIS,"OnImageUploadComplete",1)
 
*** Constrain the extensions allowed on the server
loUpload.cAllowedExtensions = "jpg,jpeg,png,gif"
 
*** Process the file or chunk
loUpload.ProcessRequest()
 
ENDFUNC

This code creates a plUploadHandler component and tells it to store files uploaded in a temp subfolder. This is a temporary folder where files are uploaded to and then discarded before getting copied to a permanent location.

We then need to map the OnUploadComplete event by mapping it to another function OnImageUploadComplete() that will do the post processing and moving of our file. Finally we can specify the file extensions that are allowed for the image and then we’re ready to Process the current request with loUpload.ProcessRequest().

This method is called multiple times for each file uploaded. Files are uploaded in chunks so a 2meg file is broken into many smaller chunks that are sent and processed one at a time. When a file is completed the OnImageUploadComplete event is fired. Here’s what that looks like:

FUNCTION OnImageUploadComplete(lcFilename, loUpload)
LOCAL lcUrl, lcFile
 
lcUrl = this.ResolveUrl("~/temp/" + lcFileName)
 
*** Resize the image
lcFile = ADDBS(loUpload.cUploadPath) + lcFileName
 
*** Delete expired files - only for 10 minutes
DeleteFiles(ADDBS(JUSTPATH(lcFile)) + "*.*",600)
 
lcNewFile =  SYS(2015) + "." + JUSTEXT(lcFile)
lcNewPath = this.cHtmlPagePath + "PostImages\" + TRANSFORM(YEAR(DATETIME())) + "\"
IF !ISDIR(lcNewPath)
    MD (lcNewPath)
ENDIF
 
lcFileName = lcNewPath + lcNewFile
*ResizeImage(lcFile,lcFileName,1024,768)
COPY FILE (lcFile) TO (lcFileName)
DELETE FILE (lcFile)
 
lcUrl = this.ResolveUrl("~/PostImages/" + + TRANSFORM(YEAR(DATETIME())) + "/" + lcNewFile)
lcUrl = "http://" + Request.GetServerName() + lcUrl
 
*** Write out the response for the client (if any)
*** In this case the URL to the uploaded image
loUpload.WriteCompletionResponse(lcUrl)
 
ENDFUNC

The handler is passed the original file name (just the filename without a path) and the loUpload component.

For the message board I want to capture the file uploaded, rename it with a random name and then move it a more permanent folder – in this case PostImages/YEAR/. Once the file has been copied the original uploaded file in the temp folder can be deleted.

Finally the OnImageUploadComplete() method has to return the new URL to the client so that the client can link to the image. If you recall in the JavaScript we were sent a response object with a response property. The response property holds whatever we write out into the WriteCompletionResponse(). The most useful thing here almost always is the full URL to the resource that was uploaded if the client allows using that resource in some way. In the client application the URL is used to embed an image link into the user’s message text.

Quite a bit of Code, but easy to do

What’s described above is the entire process involved, which is not entirely trivial. There are a fair amount of moving parts in this code both on the client and on the server, but between plUpload and Web Connection’s plUploadHandler the actual code you have to write is pretty minimal. Most of what you see above is boiler-plate code that you can cut and paste into place and then only customize the actual result handlers when uploads are complete both on the server and client. Although it’s a fair bit of code overall the non boiler-plate code is minimal.


Access-Control-Allow-Origin Header in West Wind Web Connection


No comments
Thursday, April 2, 2015, 1:44:00 AM

Over the last few weeks I’ve gotten a ton of questions for applications that need to support cross-site script access in relation to West Wind Web Connection. This typically a requirement for service applications that are accessed from mobile devices or other devices that don’t load the actual site content from the origin Web site. Typically this means you are building a HTTP or REST service that communicates with the client via JSON.

Web Connection 5.70 and later has introduced a host of new features – including the powerful and easy to use wwRestProcess Process class – to make it super easy to create HTTP based JSON services, and it looks like quite a number of users are taking advantage of these new features to integrate their Web Connection backends with various mobile device applications.

So not surprising that these requests for Access-Control-Allow-Origin header questions are coming up more frequently now. When you’re building mobile or distributed services that are called from multiple origin sites or devices you will have to address the CORS issues to enable the cross-domain Web service access.

Luckily making that happen is pretty easy.

What is CORS?

By default Web browsers (and Web browser controls) allow access of data retrieved via AJAX/XHR requests only to come from the same origin server as the originating request. This means that if I’m running on http://west-wind.com/ all of my HTTP requests using XHR have to come from the same west-wind.com domain.

The reason for this restrictive behavior is to prevent cross-site scripting attacks – if your site’s has user input that can be corrupted to include <script> tags that execute it’s possible for an attacker to potentially send sensitive information, like cookie data or anything else contained on a page to another server. The XHR restrictions are in place to supposedly prevent this.

CORS is a simple protocol allows a way to get around this by a server specifying an Access-Control-Allow-Origin header on the server to specify which domains (or all domains) that are allowed access to the site.

Setting the Access-Control-Allow-Origin Header in Web Connection

It’s easy to set this header in Web Connection:

Response.AppendHeader("Access-Control-Allow-Origin","*")

on any request that happens. Here I’m allowing any source domain to access the site. You can also specify specific domains as a comma delimited list in the format of http://west-wind.com,http://weblog.west-wind.com for example.

Alternately you can also add this globally for an entire process class by setting the value in the OnProcessInit() method:

FUNCTION OnProcessInit
 
Response.GzipCompression = .T.
Request.lUtf8Encoding = .T.
 
Response.AppendHeader("Access-Control-Allow-Origin","*")
 
RETURN .T.

Finally you can also use IIS to set this header globally for the entire site assuming your using IIS 7 or later:

<configuration>
<
system.webServer> <httpProtocol> <customHeaders> <clear/> <add name="Access-Control-Allow-Origin" value="*" /> </customHeaders> </httpProtocol> </system.webServer> </configuration>

And voila that’s all it should take to make your site able to serve remote requests from mobile devices or from pages served off a remote domain.

Things that make you go Hmmm.

Alas, I’ve never really understood the value of this CORS implementation. Cross site scripting is a big problem no doubt, but I don’t really see how CORS helps this if the *server* can specify the domain it wants to allow. If somebody is attacking your site and trying to steal information and they want to use XHR to post the data somewhere it’s trivial for the user to set up a server that adds the above header. Nothing’s been prevented at that point. This just seems like yet another useless security protocol that makes things a little bit more inconvenient for developers, but adds very little of security safeguards. Just like JSONP and hidden iFrames have been able to get around XHR cross-site limitations for years, CORS doesn’t really provide anything that prevents that.

The point of CORS is more to let the server decide of whether it wants to serve content to the client, but CORS is often billed as cross site scripting prevention which it really isn’t.


Use wwJsonSerializer to create Sample Data


No comments
Thursday, December 18, 2014, 8:43:55 PM

I frequently create demos for various applications and components and quite frequently I create objects on the fly using EMPTY objects using ADDPROPERTY(). The code that does is a bit verbose and looks something like this:

*** Note objects are serialized as lower case
loCustomer = CREATEOBJECT("EMPTY")
 
*** Recommend you assign your own ids for easier querying
ADDPROPERTY(loCustomer,"_id",SYS(2015))
ADDPROPERTY(loCustomer,"FirstName","Markus")
ADDPROPERTY(loCustomer,"LastName","Egger")
ADDPROPERTY(loCustomer,"Company","EPS Software")
ADDPROPERTY(loCustomer,"Entered", DATETIME())
 
loAddress = CREATEOBJECT("EMPTY")
ADDPROPERTY(loAddress,"Street","32 Kaiea")
ADDPROPERTY(loAddress,"City","Paia")
ADDPROPERTY(loCustomer,"Address",loAddress)
 
loOrders = CREATEOBJECT("Collection")
ADDPROPERTY(loCustomer,"Orders",loOrders)
 
loOrder = CREATEOBJECT("Empty")
ADDPROPERTY(loOrder,"Date",DATETIME())
ADDPROPERTY(loOrder,"OrderId",SUBSTR(SYS(2015),2))
ADDPROPERTY(loOrder,"OrderTotal",120.00)
loOrders.Add(loOrder)
 
loOrder = CREATEOBJECT("Empty")
ADDPROPERTY(loOrder,"Date",DATETIME())
ADDPROPERTY(loOrder,"OrderId",SUBSTR(SYS(2015),2))
ADDPROPERTY(loOrder,"OrderTotal",120.00)
loOrders.Add(loOrder)

While this works, it’s pretty verbose and a little hard to read with all the nesting and relationships. It’s still a lot less code than explicitly creating a class and then assigning properties but even so, it’s hard to see the relationship between the nested objects. The more nesting there is the more nasty this sort of code gets regardless of whether you use AddProperty() or CREATEOBJECT() with assignments.

Given that Json Serialization is readily available now, it’s actually pretty easy to replace this code with something that’s a little easier to visualize.

Take the following code which produces a very similar object by using a JSON string serialized to an object:

DO wwJsonSerializer
 
*** Create a sample object
TEXT TO lcJson TEXTMERGE NOSHOW
{
_id: "<< loMongo.GenerateId() >>", FirstName: "Rick", LastName: "Strahl", Company: "West Wind", Entered: "<<TTOC(DateTime(),3)>>Z", Address: { Street: "32 Kaiea", City: "Paia" }, Orders: [ { OrderId: "ar431211", OrderTotal: 125.44 }, { OrderId: "fe134341", OrderTotal: 95.12 } ] }
ENDTEXT
loSer = CREATEOBJECT("wwJsonSerializer") loCustomer = loSer.DeserializeJson(lcJson) ? loCustomer.LastName ? loCustomer.Entered ? loCustomer.Address.City loOrder = loCustomer.Orders[1] ? loOrder.OrderTotal

The data is created in string format and embedded inside of a TEXTMERGE statement to produce a JSON string. The JSON string is then sent through a JSON deserializer which in turn produces a FoxPro object (created from EMPTY objects and Collections).

Note that you can also inject data into this structure because of the TEXTMERGE clause that allows merging FoxPro expressions. again this works well to essentially ‘script’ your object definition.

Caveats

I use this mostly for demos and other scenarios where I need to create some static data quickly and for that it works very well because I can control the content exactly with the values I push into the object.

However, keep in mind that JSON requires data to be encoded to a certain degree. Strings need to escape extended characters, quotes and a host of other things. If you’re using Web Connection you can use the JsonString() and JsonDate() functions in the wwUtils library.

TEXT TO lcJson TEXTMERGE NOSHOW
{   
_id: "<< loMongo.GenerateId()" >>, FirstName: << JsonString(lcFirstName) >>, LastName: <<JsonString(lcLastName) >>, Entered: << JsonDate(DateTime()>>,
… }

Although this gets a little more verbose as well, it’s still easier to read than the pure FoxPro object syntax.

Visualize

Anyway – something to keep in mind. When you need more expressive syntax to construct objects – and especially complex objects – a JSON Serializer might just do the trick and provide you complex structure in a more visual way.


Visual Studio Community Edition–Visual Studio for Web Connection


No comments
Tuesday, December 16, 2014, 3:04:06 AM

A couple of months ago Microsoft announced the release of a new Visual Studio Community version to replace all the Express versions. As you probably know the Express versions of Visual Studio were severely trimmed down and specialized versions of Visual Studio  – essentially a core version of Visual Studio with only certain packages enabled. While these versions were functional and certainly provided a lot of value, each of the Express versions had some major features missing. The Visual Studio Community edition is a full version of Visual Studio Professional and is available to most non-large enterprise developers free of charge.

Web Express Limitations for Web Connection

In the past I’ve recommended using Visual Studio Express for Web, especially for developers that were taking advantage of the Web Control Framework. It provides the excellent HTML, CSS and JavaScript editors, and explicit support for WebForms which is what the Web Connection Web Control Framework uses to provide it’s designer support in Visual Studio. Express version had some limitations that definitely impacted Web Connection, namely lack of support for the Web Connection Add-in which provides the ability to bring up Visual FoxPro and your favorite browser directly from a Web Connection page.

With the intro of Visual Studio Community edition, that limitation – and many others are gone.

Visual Studio Community Edition is Visual Studio Professional

The new Community edition is essentially a free version of Visual Studio Professional for most individuals and small organizations. It provides the full feature set of Visual Studio Pro which includes the support for plug-ins, multiple projects and the whole gamut of projects that Visual Studio supports. There are no limitations or disablements – it’s a full version of Visual Studio Pro.

If you are a Web Connection developer and you’ve been using the Web Express Edition you should definitely go ahead and download and install Community Edition to get support for the Web Connection Add-in. This version also allows you to open multiple projects in the same solution so you can have multiple Web sites open in one project, as well as open the Web Connection Web Controls design time control project for creating your own components.

You can download the Community Edition from the Visual Studio Download site

Who can use the Community Edition

As mentioned the Community Edition is free to most small to medium sized organizations. With a few exceptions of larger organizations you can use Visual Studio Community edition for free. Here’s what the Community Edition site shows for who can use the Community Edition:

  • Any individual developer can use Visual Studio Community to create their own free or paid apps.

Here’s how Visual Studio Community can be used in organizations:

  • An unlimited number of users within an organization can use Visual Studio Community for the following scenarios: in a classroom learning environment, for academic research, or for contributing to open source projects.
  • For all other usage scenarios: In non-enterprise organizations, up to 5 users can use Visual Studio Community. In enterprise organizations (meaning those with >250 PCs or > $1 Million US Dollars in annual revenue), no use is permitted beyond the open source, academic research, and classroom learning environment scenarios described above.

What’s the Downside?

I think that the Community Edition is a huge step in the right direction for Microsoft. Developer tools are becoming more and more of a commodity item and are used more to sell a platform than anything else, and this edition at least provides a full featured development tool for those that want to work with Microsoft technologies. The fact that the HTML, CSS and JavaScript editors are top notch for any kind of Web development is an additional bonus.

The one downside is that Visual Studio is large. It uses a fair amount of system resources so you need a reasonably fast machine to run it. However, it’s nothing like older versions of Visual Studio that were. If it’s been few years since you’ve last tried Visual Studio you really should give it another shot as performance and especially editor performance and load times have improved significantly.

As a Web Connection Developer Should you care?

If you’re using the Web Control Framework you will definitely want to use the Community Edition as it gives you full support for the designer integration and the Web Connection Add-in to load your WCF pages. If you’re using any other edition, go make the jump – it’s totally worth it.

But even if you’re not using the Web Control Framework that provides integration with the WebForms designer, there are many features in Visual Studio’s HTML, CSS and JavaScript editors that are top notch if not the best compared to just about anything out there. The only other tool I would put even close in comparison is Jetbrains WebStorm which is an excellent tool for pure HTML5 and JavaScript (including Node) based applications.

If you are using any version of Visual Studio 2012 or 2013 (including the Community Edition) make sure you install Web Essentials which provides additional HTML, CSS and JavaScript features including support for LESS, SASS, Zen Coding support (very efficient HTML shortcut template language), automatic JS and CSS minification and much more. As the name implies it’s essential and I don’t consider this an optional add-in.

Regardless of whether you do Web Connection development, or raw HTML coding, here are some features in Visual Studio that I wouldn’t want to live without:

  • HTML Element and Attribute IntelliSense
  • HTML and CSS Formatting (as you type or explicit)
  • CSS IntelliSense (CSS properties, reference props in docs)
  • F12 Navigation to CSS classes
  • Automatic LESS parsing and saving
  • Excellent JavaScript IntelliSense (especially with _references.js file)
  • Basic JavaScript Refactoring

Web Essentials:

  • Zen Code
  • Script and CSS Compression
  • HTML Color Picker, Darken/Lighten
  • CSS Browser Support Preview
  • CSS Vendor Prefix injection
  • Font and Image Previewers

There’s lots more but those are some of the highlights for me. If you haven’t tried Visual Studio in a long while or where put off by the pricing of the full versions, give the community edition a try…


SSL Vulnerabilities and West Wind Products


No comments
Wednesday, October 15, 2014, 6:32:36 PM

Several people have sent me frantic email messages today in light of the latest POODLE SSL vulnerability and whether it affects West Wind products.

The vulnerability essentially deals with older SSL protocol bugs and the concern is especially on the server that protocol fallback from TLS to SSL can cause problems. Current protocols are TLS vs. the older SSL technology. Only the older SSL stacks are vulnerable. This is most critical on Web Servers, and you'll want to disable the older SSL protocols on IIS and leave only TLS enabled. TLS is the newer standard that superseded  SSL, although we still talk about SSL certificates – most modern certificates are actually TLS certificates and have support for SSL for fallback validation.

You can check your server’s SSL certificate status here:
https://www.ssllabs.com/ssltest/index.html

You’ll want to see that all versions of SSL are either disabled or at the very least that the server doesn’t support protocol fallback to SSL protocols.

When I ran this for my site running Server 2008 (IIS 7.0) I found SSL3 enabled, but the server not supporting automatic protocol fallback which avoids the main POODLE issue. I’ve since disabled SSL V3 on the server. The good news is, that while the certificate isn’t using maximum strength, the server was not found to be vulnerable from this issue even with the default settings in place.

Disabling SSL on IIS

If you see that SSL versions are enabled even if you have TLS certificates (which just about all of them should be by now) you can disable specific protocols. This is done through registry keys, but be aware that this is global for the entire machine. You can disable/enable protocols for both the client and server.

Here's a KB article from Microsoft to check out that tells you how to disable older SSL protocols.
http://support.microsoft.com/kb/187498

SSL and WinInet

The West Wind Client Tools and specifically the wwHTTP class rely on the Windows WinInet library to provide HTTP services. The POODLE issue is much less critical for the client as the client is the actually attacking entity. But I double checked here on my local machine and I can see that WinInet uses TLS 1.2 on my Windows 8.1 when connecting to a certificate on the server.

Capturing an https:// session from Fiddler shows the following request header signature on the CONNECT call:

Version: 3.3 (TLS/1.2)
Random: 54 3F 15 D6 B5 3E 6B F5 AD 71 41 FB 4C 39 B9 30 C5 21 04 A4 76 7F 87 A5 1A BA D6 83 19 B3 10 3B
SessionID: empty

which suggests TLS/1.2 as the initial request sent.

I don’t know what older versions of Windows – XP in particular use – but I suspect even XP will be using TLS/1.0 at the very least at this point. Maybe somebody can check (Use Fiddler, disable Decrypt HTTPS connections option, then capture HTTPS requests and look at the raw request/response headers.

Nothing to see here

From the West Wind perspective this issue is not specific to the tools, but to the operating system. So make sure the latest patches are installed and that if you have to remove the server SSL certificates. Client software is not really at risk, since the attack vector is a receiving Web Server. Regardless even the client tools appear to be using the latest safer protocols and so any West Wind tools are good to go.


Figuring out Web Connection Versions


No comments
Friday, September 26, 2014, 7:40:23 PM

On various occasions I’ve gotten questions regarding how to figure out what version of Web Connection is running on the server. ‘Web Connection’ has several components so the version numbers may be separate and while it’s not absolutely critical that all components are on the same versions it’s a good idea to keep them in sync.

Connector Versions

Web Connection has three different connector versions:

  • Web Connection .NET Module (recommended for IIS 7 and later)
  • Web Connection ISAPI Module
  • Web Connection Apache Plug in

For most applications I’d recommend using the .NET module as it has the most features and is the most reliable, especially when running under COM since it has a high performance managed threadpool for managing the COM instances.

Each of these components has their own signature, but there are several ways you can check for versions:

  • Check the actual DLLs
  • Check the Status Form after a Request
  • Check the Admin ISAPI Admin Form

Check the actual DLLs

The DLLs are stamped with a Version attribute that indicates the actual version. To check the version you can right click the file and use Properties | File Details to get to the embedded resource data that’ll contain the version number. The .NET module and ISAPI modules are updated regularly. You can find the version numbers in either dll right clicking and using File | Details:

WebConnectionModuleFileVersion

Check Web Connection Status After a Request was fired

You can also check the version after at least one request was fired by using the Status form and then reviewing the request data.DllVersion

This is a good way to see if you’re getting the right DLL file. It’s also a good place to check the wcConfig path, which will tell you which configuration is being read which indirectly also tells you where the DLL lives (in the same folder for wc.dll or the bin folder for webconnectionmodule.dll).

Check the Web Connection Module Administration Page

The Web Connection Module page also shows the full version of the active DLL. The module screens looks slightly different depending which version of the DLL you are running – the .NET Module or the ISAPI component. But both contain the version number and status information. For the module it looks like this:

WebConnectionStatusVersion

For the ISAPI page the DLL version shows (ISAPI) in parenthesis instead.

FoxPro Server Version

Note that the Module page also displays the compiled server’s EXE version number. This is useful to let you know what version your own code is. Note this is only avaiable with the module and not with the ISAPI component.

FoxPro Code Versions

To figure out the Web Connection code versions you can easily check the version number in wconnect.h. The version number in there shows up like this:

#DEFINE WWVERSION "Version 5.69"
#DEFINE WWVERSION_NUMBER 5.69
#DEFINE WWVERSIONDATE "September 26th, 2014"

This is the only FoxPro code based version that Web Connection provides – individual components are not versioned directly, so if you make changes to any of the Web Connection code these version numbers may not be accurate.

Keeping Versions Straight

These days I get a lot of email in regards to old versions that still float around and lots of people are asking what versions they are running and how various components can interact and work with newer version framework code.

The short answer is that the update from .NET 4.x to 5.x is not major, but does require some changes. Updates from Version 5.0 to 5.50+ is almost seamless. There are very few changes on the latter upgrade path.

Smaller version changes are generally just small maintenance releases and I would recommend that from time to time you update to the latest version just to keep up to date. Newer versions tend to fix reported bugs, improve performance for some operations and are maintained for security and other critical changes that might be required. It’s probably not necessary to update to every new version that comes along but I suggest that at least once or twice a year you update your code to the latest. Partial version updates are free (5.50+ versions) and it’s a good idea to keep your code up to date to minimize future update pain. It’s easier to fix one or two minor items at a time, the 20.

I hope this addresses some of the questions that have come my way regarding versioning and on how to best keep your Web Connection version up to date.


Creating .DLL Script maps with the Web Connection .NET Managed Module


No comments
Sunday, July 20, 2014, 12:49:01 AM

On IIS 7 and later, Web Connection’s preferred connector interface is the .NET connector, which internally uses an ASP.NET HttpHandler implementation to interface between IIS and the Web Connection server. Functionality wise the .NET connector has the same feature set as the ISAPI one, plus some additional features, but the main reason it’s the preferred choice is because it’s much easier to set up with the default configuration of IIS, and – surprisingly it actually offers better performance and stability than the ISAPI implementation especially in COM mode.

The managed module also works with IISExpress out of the box, so in a development environment you can easily use a non-system level tiny IIS Express implementation that’s easy to run and configure vs. having to deal with the full IIS environment configuration (which can be a bit daunting).

Lots of wc.dll Apps still around

I do quite a bit of IIS configuration support these days – a lot of people are finally upgrading ancient servers to newer versions of Windows, typically Windows 2012 these days. One issue that has kept some people from updating to the .NET managed module has been, that especially older Web Connection applications, are accessing the raw wc.dll directly as part of the URL. I’ve long recommended this as a bad practice, because accessing the ISAPI DLL is both a potential configuration issue as IIS makes it hard to access DLLs directly and in some cases disallows it altogether. But nevertheless there are a lot of old applications out there that still use direct wc.dll links. Direct DLL links also make it much more difficult to deal with paths as you always have to reference a physical location for the dll – script maps are much more flexible as they can be called from anywhere so path dependencies just go away in many cases.

Using a *.dll Script Map with the .NET Managed Module

So what if you have a complex application and you can’t (or won’t) give up the wc.dll links in your application?

Today I was working with a customer through some configuration issues. We started discussing the application, and as I always do I recommend using the .NET module instead of ISAPI. We went over the improvements and why it’s great, and we both agreed that this would be great, but what about our .DLL extensions in the URL?

After some problems with the ISAPI configuration, I actually went ahead and set up the .NET Handler configuration to ascertain that things were working and running and sure enough with the .NET module things were just working. Since it worked with the module -in a flash of enlightenment - we decided to try to create a script map for .DLL and point it at the Web Connection .NET handler.

And lo and behold – the .dll Script mapping mapping worked perfectly!

It’s absolutely possible to create .DLL script map to a .NET Managed handler, which makes it possible to run existing Web Connection applications that use wc.dll directly on the .NET managed module.

Here’s what the relevant web.config settings look like (or you can use the IIS Admin interface to create this as well):

<configuration> 
<system.webServer> <!-- use this on IIS 7.5 and later --> <httpErrors existingResponse="PassThrough" /> <!-- IIS 7 Script Map Configuration --> <handlers> <add name=".wc_wconnect-module" path="*.wc" verb="*"
type="Westwind.WebConnection.WebConnectionHandler,WebConnectionModule" preCondition="integratedMode"/> <add name=".dll_wconnect-module" path="*.dll" verb="*"
type="Westwind.WebConnection.WebConnectionHandler,WebConnectionModule" preCondition="integratedMode"/>
</handlers> </system.webServer> </configuration>

When creating a script map to a .NET handler a  DLL script handler is just like any other script map. Using the mapping you can run requests like this:

http://localhost/wconnect/somepage.dll

Ok that looks weird and is probably not your typical use case, but it actually works. More interestingly though you can do this:

http://localhost/wconnect/wc.dll?_maintain~ShowStatus

When this page comes up you’ll see that it loaded the .NET Managed handler, even though we referenced wc.dll! For this to work – you’ll want to make sure you remove the physical wc.dll from disk and have the webconnectionmodule.dll in the bin folder of your site.

This means that if you have an existing application that used any wc.dll links directly you can now use them with the .NET handler. Additionally you can start to clean up your application to use scriptmaps instead of the DLL link in the first place.

Why you should use ScriptMaps instead wc.dll Links

As already mentioned ISAPI configuration is getting more and more tricky as IIS versions progress. IIS today mainly relies on .NET to handle extensibility to other services and interfaces and ISAPI is more of an deprecated prototocol. It’s still there but it’s an optionally installed feature. Further  if you are accessing a DLL directly using ISAPI (not through a scriptmap) you are directly accessing a binary file which is generally discouraged. In order for this to work you have to explicitly enable generic ISAPI extensions in the IIS configuration.

Scriptmaps are simply a nicer way to write URLs. Instead of the ugly ~ syntax of:

wc.dll?ProcessClass~Method~&Action=Edit

you can use scriptmap to method mapping:

Method.sm?Action=Edit

where sm is a scriptmap, and method is the method of the process class that sm is mapped to. The URLs are much cleaner and easier for users to parse and understand.

Finally script maps allow you to simplify relative paths. Because a script map is not referencing a physical file in a specific folder like wc.dll, you can reference a scriptmap from any location and have it behave the same way. This means if you create a page in a subdirectory and you want to access the scriptmapped page you can use the same path and it will work. IOW:

Method.sm?Action=Edit
Admin\Method.sm?Action=Edit
Account
\Method.sm?Action=Edit

all are treated exactly the same. The only difference is that the script path passed as part of the server variables will point to a different location, so Request.GetPhysicalPath() will point at the site relative physical path on disk. Otherwise each of those three commands are identical.

Scriptmaps are the way to go!

Scriptmaps make life easier and once again I urge you, if you’re not using them to think about integrating them in your Web Connection applications. I suspect sometime in the not too distant future IIS will stop supporting direct access .DLL links and will force all operation to occur against script maps. Plus the easier usage for referencing dynamic links make code much more portable across sites or virtual directories if your development and live environment aren’t 100% identical.


UTC Time in FoxPro


No comments
Monday, June 9, 2014, 11:07:46 PM

Representing dates and times across timezones can be a challenge especially if you don’t lay out a plan up front on how to store dates consistently. The sneaky thing with date time management in larger applications and especially applications that live on the Web or are shared across locations, is that problems don’t usually show up until much later in the lifetime of the application. For FoxPro in particular it’s not natural to store dates in anything but local machine format as the language doesn’t support direct UTC formats so it’s very common to see FoxPro applications use local dates which is usually a bad idea.

Here’s why and how we can address these issues…

TimeZones and Offsets

Depending on where you are in the world your local time is defined by an offset from UTC (Coordinated Universal Time) time or the baseline zero time. As you know if you’ve ever talked to somebody half way across the world at a certain time of day, while you just got done with breakfast, they are getting ready to go to bed on the other side of the world. This is the timezone offset. If you build applications that deal with customers that enter data into a system from multiple locations then using local times becomes problematic.

The problem of local times is made worse by Daylight Savings time. Most of the world – especially those further away from the equator – have daylight savings time which is applied on different dates in different locations around the world. Usually it’s “spring forward” and “fall back” with time getting set one hour forward for the summer.  Some countries have it, others don’t, and surprisingly – some countries actually half hour DST offsets. You can see where this is going – dealing with dates from multiple locations around the world can get complicated fast.

UTC Dates

A generally accepted solution to this problem is to store date values using a single time format that is adjusted from local time. Typically this date format is UTC time – or zero offset time.

I often get questions about why you should – and you REALLY, REALLY should - store dates in UTC format. The simple reason is: Things change! You never know how the data that you are capturing today will be used in the future. Maybe today you’re using the data in one location but maybe in the future you have a Web application and you have multiple locations that access the data. Or you end up building a service for other people to consume your data. If your data is in local time, the data will be much less useful then being in a universal format.

While it’s possible to convert data later on.

In fact on this project I worked on the client insisted on going with local dates over my vehement protests. The argument almost always is that ‘hey, we just have one location – everybody’s running in this domain and we want to see data from the database in local time.’ Convenient – yes absolutely. A good idea: NEVER! In every application where this has come up for consideration it’s always caused a problem eventually. Yes you may not see this right away because when you choose to stick with local time it’s usually based on the assumption you’re staying with inputs from a single timezone. Over time, as applications age however, things change. Data is accessed in other ways, possibly from different applications or shared with other customers around the world. And all of a sudden you have a problem that you never thought would happen.

It’s a generally accept good software practice to store Date and Time values in a consistent format and the easiest way to do this is to use UTC dates. The idea is simple: All data that is persisted to a permanent store is turned into UTC dates and written out that way. Any data retrieved is converted into a local date explicitly – only for display purposes. For things like queries input dates that are locale specific are converted into UTC dates first before the query is applied. IOW, if you use a common date format there will be conversion, but typically only when accessing/querying the data from the UI.

Most software systems provide easy support for date conversions. In .NET for example, there’s a DateTime.UtcNow value you can use to get the UTC time and there FromUniversalTime() and ToUniversalTime() and ToLocal

FoxPro and Date and Time

FoxPro doesn’t make this easy because it can only represent dates in local time – that is the time that is current for the computer that the machine is running on. However, it’s quite common in other environments such as .NET and Java to always write out date time values as UTC time. UTC time is Zero time, Greenwich (England) time, Zulu time – whatever you want to call it, it’s the time that doesn’t have an offset.

At the very least if you need to interact with systems that use UTC time you’ll need to make FoxPro play nice in this space. But I would urge you to consider to ALWAYS use UTC time for applications. While it is definitely a little more work to deal with UTC deformatting it’s not that much effort as long as you realize that the only time you care about this is when you convert dates to and from the User interface. Internally all date operations can relatively easily be made with native UTC.

Some UTC functions for FoxPro

If you are using any of our West Wind products – West Wind Web Connection, West Wind Client Tools or Internet Protocols – you already have this functionality I’m going to describe below. It’s built in with two functions (contained in wwAPI.prg):

GetUtcTime(ltTime)
Gets the current UTC time, or converts a FoxPro local DateTime to a UTC time.

FromUtcTime(ltTime)
Converts a date in UTC time format to local time.

Additionally there’s also:

GetTimeZone()

Returns the current timezone offset from UTC for the local machine. This is useful if you DIDN’T use UTC dates and are later forced to adjust dates based on local time and calculating time offsets based on user options or external locale access (ie. over a service). Essentially what this allows you to do is calculate relative offsets between two timezones and calculate a time for a different timezone. This function is also used by GetUtcTime() and FromUtcTime().

 

If you’re wondering about the inconsistent naming – the original function that existed in the framework for years was GetUtcTime which simply returned the current UTC time. Then at a later point I added the functionality to arbitrarily convert any DateTime value to a UTC data, so the function name stayed.

So using the two Utc conversion functions you can do the following:

? "Timezone: " + TRANSFORM(GetTimeZone()) +  " minutes"
ltTime = DATETIME()
? "Current Time: ", ltTime
 
ltUtc = GetUtcTime(ltTime)
? "UTC Time: ", ltUtc
 
ltTime = FromUtcTime(ltUtc)
? "Back to local: ", ltTime

I’m currently in the PDT (Pacific Daylight Time) zone and I get:

  • 06/09/2014 07:45:36 PM  - current
  • 06/10/2014 02:45:36 AM  - UTC
  • 06/09/2014 07:45:36 PM  - back to current
  • 420   -  timezone offset in minutes (-7 hours)

Note that GetTimeZone() will change if you change your system timezone, but VFP doesn’t see the change until you restart. The GetTimeZone() value also seems backwards: It’s +420 for Portland Oregon (PDT)  and –600 for Sydney Australia, but that’s how the Windows API is actually returning it. Essentially you can add the GetTimeZone() value to a local date to get a UTC date.

Implementation

This is all nice and neat if you have West Wind tools, but what about the rest of you that don’t? Ok, here’s some code that provides this same functionality (or pretty close to it actually):

*** Code exists also in wwAPI of any West Wind Tools!
*** SET PROCEDURE TO wwAPI Additive
#define Testing .t.
 
SET PROCEDURE TO TimeZone additive
 
#IF Testing
 
? "Timezone: " + TRANSFORM(GetTimeZone()) +  " minutes"
ltTime = DATETIME()
? "Current Time: ", ltTime
 
ltUtc = GetUtcTime(ltTime)
? "UTC Time: ", ltUtc
 
ltTime = FromUtcTime(ltUtc)
? "Back to local: ", ltTime
#ENDIF
 
 
************************************************************************
*  GetUtcTime
****************************************
***  Function: Returns UTC time from local time
***    Assume:
***      Pass:
***    Return:
************************************************************************
FUNCTION GetUtcTime(ltTime)
 
IF EMPTY(ltTime)
    ltTime = DATETIME()
ENDIF
 
*** Adjust the timezone offset
RETURN ltTime + (GetTimeZone() * 60)    
ENDFUNC
*   GetUtcTime
 
************************************************************************
*  FromUtcTime
****************************************
***  Function: Returns local time from UTC Time
***    Assume:
***      Pass:
***    Return:
************************************************************************
FUNCTION FromUtcTime(ltTime)
RETURN ltTime - (GetTimeZone() * 60)
ENDFUNC
*   FromUtcTime
 
************************************************************************
FUNCTION GetTimeZone
*********************************
***  Function: Returns the TimeZone offset from GMT including
***            daylight savings. Result is returned in minutes.
************************************************************************
PUBLIC __TimeZone
 
*** Cache the timezone so this is fast
IF VARTYPE(__TimeZone) = "N"
   RETURN __TimeZone
ENDIF
 
DECLARE integer GetTimeZoneInformation IN Win32API ;
   STRING @ TimeZoneStruct
   
lcTZ = SPACE(256)
 
lnDayLightSavings = GetTimeZoneInformation(@lcTZ)
lnOffset = CharToBin(SUBSTR(lcTZ,1,4),.T.)
 
*** Subtract an hour if daylight savings is active
IF lnDaylightSavings = 2
   lnOffset = lnOffset - 60
ENDIF
 
__TimeZone = lnOffset
    
RETURN lnOffSet
 
************************************************************************
FUNCTION CharToBin(lcBinString,llSigned)
****************************************
***  Function: Binary Numeric conversion routine. 
***            Converts DWORD or Unsigned Integer string
***            to Fox numeric integer value.
***      Pass: lcBinString -  String that contains the binary data 
***            llSigned    -  if .T. uses signed conversion
***                           otherwise value is unsigned (DWORD)
***    Return: Fox number
************************************************************************
LOCAL m.i, lnWord
 
lnWord = 0
FOR m.i = 1 TO LEN(lcBinString)
 lnWord = lnWord + (ASC(SUBSTR(lcBinString, m.i, 1)) * (2 ^ (8 * (m.i - 1))))
ENDFOR
 
IF llSigned AND lnWord > 0x80000000
  lnWord = lnWord - 1 - 0xFFFFFFFF
ENDIF
 
RETURN lnWord

The code is pretty self-explanatory. GetTimeZone() makes a call to a Windows API function to retrieve the Timezone structure and then needs to do some binary conversion to peel out the timezone offset. The timezone value is cached so only the first call actually makes the API call for efficiency.

Again if you are already using any West Wind tools you won’t want to use this code as it’s already included, but if you don’t then these functions are feature compatible with the West Wind Versions.

Working with UTC Dates

Using UTC dates in your application is pretty straight forward. Your user interface captures dates and times as local datetime values as it always has, but when you actually write the data to the database you convert the date to UTC dates before writing them.

Note that you do have to be somewhat careful to ensure dates are always normalized. FoxPro has no concept of a date kind – strongly typed languages like .NET and Java actually treat dates as structures that contain additional information that identify the date type. So if you tried to convert a date to UTC that is already a UTC kind it won’t convert again and hose that object. Some languages are even better about his: JavaScript stores all dates as UTC dates, and only the string functions that convert or print dates actually convert the date to local time (by default). Other overloads allow getting the raw UTC dates out. This is actually an ideal case – you get safe date values and the language itself drives the common use case that dates are used as local dates at the application UI level.

Unfortunately in FoxPro there no safeguards for this situation as dates are always local dates with no built in way to convert. So it’s up to you to make sure that you know which format a date is stored in.

When running queries against the data on disk with dates input by users or from other sources that are in local date format, you first convert the input dates to UTC dates, then run your queries with the adjusted date values:

*** some date that comes from the UI
ltUserDate1 = DATETIME()
ltUserDate2 = DATETIME() - 3600 * 24 * 30
 
ltTo = GetUtcDate(ltUserDate1)
ltFrom = GetUtcDate(ltUserDate2)
 
SELECT * FROM orders ;
  WHERE OrderDate >= ?ltFrom AND OrderDate <= ?ltTo
  into cursor Orders

Likewise if you write data to disk that was captured from user input you have to capture the local date and convert it. If you’re displaying value you have to convert them to local dates. If you’re using business objects, you can do this as part of the business object’s save operation which can automatically update dates as they are saved. Properties can have setters and getters that automatically convert dates to the right format.

Typically this will be a two step process – loading and saving.

For saving you might do:

*** Save Operation
loOrderBus = CREATEOBJECT("Order")
loOrderBus.New()
 
loOrder = loOrderBus.oData
 
IF EMPTY(loOrder.Entered)
    loOrder.Entered = GetUtcDate()       
ELSE    
    loOrder.Entered = GetUtcDate(loOrder.Entered)
ENDIF

and for reading you would do the opposite:

*** Load Operation
loOrderBus = CREATEOBJECT("Order")
loOrderBus.Load(lcOrderId)
 
loOrder = loOrderBus.oData
 
IF !EMPTY(loOrder.Entered)
    loOrder.Entered = FromUtcDate(loOrder.Entered)
ENDIF

If you’re using business objects like the above you can make this even more transparent by automatically doing these transformations right inside of the business object itself:

DEFINE CLASS busOrder as wwBusiness
 
FUNCTION Load(lcId)
 
IF (!base.Load(lcId)
  RETURN .F.
ENDIF
 
this.oData.Entered = FromUtcDate(this.oData.Entered)
 
RETURN .T.
ENDFUNC
 
FUNCTION Save()
this.oData.Entered = GetUtcDate(this.oData.Entered)
RETURN base.Save()
ENDFUNC
  
ENDDEFINE  

which makes the assumption that your user code deals with local timezones while the data saved is UTC.

Dated

Clearly all of this isn’t just totally transparent, even in languages that support UTC more easily there’s some effort involved to make this work. The main reason being that database – the storage mechanism in most cases doesn’t differentiate between dates either. FoxPro data doesn’t, neither does SQL Server. NoSQL solutions like Mongo do because they’re using JSON values which are ALWAYS UTC dates – you don’t get a choice (which in my opinion is the right way).

It’s not a totally transparent process, but it’s a good idea to do this nevertheless especially if you’re building applications that run on the Web or in other places where the applications are accessed from multiple locations – which is most applications these days. It’s worth the effort for peace of mind in the future and a good skill to learn as this is the norm for other platforms that are more date aware than FoxPro.


Showing a Wait Message when submitting longer Requests


No comments
Tuesday, February 11, 2014, 1:49:33 PM

Here’s a question that comes up quite frequently: I’m running a backend request that takes a few seconds to run – how do I show a simple ‘wait’ message while this request runs?

There are a number of ways to approach this problem including some more complex async processing concepts that offload processing to a separate process to avoid tieing up Web Connection instances that are running. But, if your process is a one off process that rarely runs and doesn’t take a huge amount of time (my cutoff is usually in the 10-20 second range before I’d consider async operations), there’s a simple UI solution that might make waiting a little easier to deal with.

It’s the UI Stupid

The problem with long running submit requests is that when the user clicks a button to submit a form, they likely expect something to happen right away. If your app however, just sits around waiting for a response to come back, the user is very likely to think that something is wrong. The most likely response will be that they either wander off without waiting for completion, or in a typical UI will try to click the Submit button again. For the latter, where you had one small problem before, you now have multiple small problems that can very easily tie up your server because those requests are slow.

So a UI solution to this should address two things:

  1. Providing some sort of UI that lets the user know that something’s happening
  2. Prevents the user from clicking the submit button again

It turns out doing this is pretty simple. Let’s look at an example. The following is a simple form that has a submit button.

SubmitBasicForm

I’m using a Web Connection Web Control Framework button here, but the same rules apply if you’re using Web Connection templates or script and plain HTML controls – the WCF just makes the code more self-contained.

<form id="form1" runat="server">
    <ww:wwWebErrorDisplay runat="server" id="ErrorDisplay" />

    <div class="contentcontainer">
        <h2>Slow Submit Operation</h2>
        
        <p>
            The following button click will take a few seconds to run.
            If your code doesn't do anything, the application will appear
            to simply be unresponsive but the existing form is still active
            daring the user to click the submit button again... and again...
            and again.
        </p>        
        
        <hr />
        <ww:wwWebButton runat="server" id="btnSubmit" Text="Submit to Server" 
                        CssClass="submitbutton"  Click="btnSubmit_Click"/>

    </div>
</form>

On the server side the FoxPro code in the button simply waits for a few seconds and dumps a message to the ErrorDisplay control on the page.

FUNCTION btnSubmit_Click()
   WAIT WINDOW "Hold on - processing..." TIMEOUT 10 
   this.ErrorDisplay.ShowMessage("Waited for 10 seconds.")
ENDFUNC|

This forces the server to essentially ‘hang’ for 10 seconds to simulate some long running task.

If you run this now by hitting the Submit button, you’ll see that the UI doesn’t change after the submit button is hit. The HTML DOM pointer changes to a spinning pointer, but otherwise the UI stays the same. The Submit button is still active so you can actually press it again, and if you do you generate another hit against the server.

Ok, so this UI works for quick requests that process in under 1 second, but for anything over a few seconds this becomes problematic.

Using JavaScript to show Status

The HTML DOM has a form submission event that fires when an HTML form is submitted. When you click the Submit button the form.submit() event is fired and we can capture this event to modify the browser UI by showing a progress dialog.

I’m going to be using the Web Connection client modal dialog control that makes this super easy. Modal dialogs show a black overlay on the page and overlay whatever content you apply it to ontop of this black overlay. The overlay basically prevents access to the underlying document – so you can’t click the submit button again – and provides nice visual feedback that the content is not accessible. You can then overlay some content of your choice.

Here I use a rotating image gif and some text. Here’s what it looks like:

ModalOverlay

The markup for this simple HTML looks like this and I simply add this to the bottom of the HTML page.

    <div id="WaitDialog"   class="hidden" style="text-align: center">
        <img  src="css/images/loading_black.gif" />
        <div style="margin-top: 10px; color: white">
            <b>Please wait</b>
        </div>
    </div>

Note this uses the westwind.css stylesheet for the hidden CSS class and the loading_black.gif from the CSS folder.

Now to hook this all up we can use just a few lines of JavaScript:

<script src="scripts/jquery.min.js"></script>
<script src="scripts/ww.jquery.min.js"></script>
<script>
    $("#form1").submit(function() {
        $("#WaitDialog").modalDialog();
    });
</script>

I’m loading jQUery and ww.jquery.js from the scripts folder. jQuery provides the base functionality for document parsing and ww.jquery.js provides the modal dialog feature (and a lot more modular functionality built-ontop of jQuery).

The .modalDialog() jQuery plug-in is attached to the element you want to display centered on the screen on top of the darkened background. Here I’m using the default modal dialog behavior without any options parameters, but there are a host of options available to customize the modal popup and you can see some more examples on the Web Connection sample site.

Summary

Using a modal overlay like this is a simple solution to longish but not overly long server side requests where you want to prevent users from just walking off or double clicking buttons. Capturing the form submission can also be even simpler – for example you could simply disable the submit button, or change the submit button text into something like “Processing…” and disabled rather than displaying the overlay which is a good idea for shorter requests where it might be jarring to have a modal dialog pop up and then disappear immediately.


wwFtp and WinInet Problems with IE 11


8 comments
Monday, November 25, 2013, 4:49:20 PM

In the last few weeks I've gotten a few notes from various customers that wwFtp has started to break for them after upgrading to IE 11. It appears there's been a new bug introduced with IE 11 that causes failures on certain types of connections.

There's more info on this in this StackOverflow question:

http://stackoverflow.com/questions/19683291/wininet-from-ie-11-randomly-returns-error-12003-for-most-ftp-functions

Specifically it points at some reproducible problems.

So far all the problems I've heard reported are related to file uploads rather than downloads. And for uploads there are a few workaround available.

Setting the Scenario

So I set up a simple test to upload some files to my own FTP server and sure enough I can see the problem occurring with this code:

CLEAR
DO wwftp
DO wwutils
 
LOCAL o as wwFtp
o=create("wwFTP")

? o.FTPConnect("www.west-wind.com","ricks",GetSystemPassword())
? o.FTPSendFileEx("c:\temp\geocrumbs.jpg","geocrumbs.jpg")
? o.FTPSendFileEx("c:\temp\iefullscreen.png","iefullscreen.png")
? o.FTPSendFileEx("c:\temp\iemetrononfullscreen.png","iemetrononfullscreen.png")
? o.cErrorMsg
o.FTPClose()
 
RETURN

Here I'm using FtpSendFileEx() to upload multiple files on a single connection and when I run this the third FTPSendFileEx call ends up failing with a 12003 error (extended error) with an error message basically echoing back the transfer command.

200 Type set to I
226 Transfer OK

Oddly this looks like the transfer worked even though the error message shows, but examing the target folder on the server reveals that the file was not actually sent or at least not written over there.

So what can we do?

Disable lPassiveFtp

wwFtp by default uses Passive FTP as it's more flexible for going through firewalls and proxied HTTP connections. By default lPassiveFtp is .T. and so passive FTP is used. Passive FTP basically creates a connection does it's transfer and drops the connection and automatically reestablishes as needed. In the process the FTP connection might actually jump ports.

By setting:

o.lPassiveFtp = .F.

wwFtp uses Active connections which stay alive for the duration of the connection made and stays fixed on a single port.

Using Active Connections I had no problems with uploading many files.

Unfortunately, active connections do not always work and can be flaky with connection drop offs, but it entirely depends on the environment. In general I recommend using Passive as the default but in light of the current bug, using Active at least should be tried (and you should always expose that setting as an option in your application settings so you can easily switch modes.

Connect and disconnect for each Transfer

Another option for sending files is to simply not reuse connections and send files by opening, sending and closing the connection. Doing this is reliable but it'll add a little overhead especially if you're sending lots of files.

So this works too:

LOCAL o as wwFtp
o=create("wwFTP")
o.lPassiveFtp = .T.
 
? o.FTPConnect("www.west-wind.com","ricks",GetSystemPassword())
? o.FTPSendFileEx("c:\temp\geocrumbs.jpg","geocrumbs.jpg")
o.FTPClose()
? o.FTPConnect("www.west-wind.com","ricks",GetSystemPassword())
? o.FTPSendFileEx("c:\temp\iefullscreen.png","iefullscreen.png")
o.FTPClose()
? o.FTPConnect("www.west-wind.com","ricks",GetSystemPassword())
? o.FTPSendFileEx("c:\temp\iemetrononfullscreen.png","iemetrononfullscreen.png")
o.FTPClose()

Use FtpSendFileEx2

Some time ago I quietly added an wwFtp::FtpSendFileEx2() function to wwFtp. The original FtpSendEx() method is a very low level function that makes a bunch of internal API calls from within Visual FoxPro. It also adds some additional features like giving you the ability to get notified of chunks of data being sent.

Even prior to this IE 11 issue, I've found that on occasion when sending large numbers of files, FtpSendFileEx() would occasionally stall for no apparent reason. It was rare but enough of a problem to consider alternatives. The alternative was to build another routine - FtpSendFileEx2(), which provides the same functionality but calls one of the higher level WinInet functions (FtpPutFile()) which is basically a single command. It turns out that using FtpPutFile() under the hood is a lot more reliable than the various API streaming functions.

FptSendFileEx2() is parameter compatible with FtpSendFile() but - it doesn't support the update even calls the OnFtpBufferUpdate() to provide progress information as the file is sent off a single API call into WinInet.

LOCAL o as wwFtp
o=create("wwFTP")
 
? o.FTPConnect("www.west-wind.com","ricks",GetSystemPassword())
? o.FTPSendFileEx2("c:\temp\geocrumbs.jpg","geocrumbs.jpg")
? o.FTPSendFileEx2("c:\temp\iefullscreen.png","iefullscreen.png")
? o.FTPSendFileEx2("c:\temp\iemetrononfullscreen.png","iemetrononfullscreen.png")

o.FTPClose()

and that works reliably too.

I use FtpSendFileEx2() in Help Builder to upload HTML help files to a Web site, and that can be 1000s of files in a session - and it works without problems, so this is what I would recommend you use for uploading files in bulk.

What to use?

If you're uploading a bunch of smallish files a use FtpSendFileEx2() - you won't need progress info on a per file basis, but you can certainly handle intra file upload info. If you upload just one or two larger files and you need the OnFtpUpdate() API to provide progress info, use FtpSendFileEx() but just make sure you reconnect for each file upload.

Switching active/passive mode is just a quick fix that might help get you out of a bind, but as a long term solution I'd still recommend you use Passive mode as it's much more reliable.

Hopefully this will be a temporary issue that Microsoft addresses soon - this is turning out to be a major headache for some of my customers who've been calling in frantically asking to see if there's a solution to this problem. It sucks when an application that's been running for 10 years mysteriously breaks after a silly browser update.




© Rick Strahl, West Wind Technologies, 2004 - 2015