Delphi Developers
Components for Internet Component Suite (ICS)

6th July 2021 - Updated PEM Bundle CA Trusted Store Files

1st April 2021 - ICS V8.66 released includes support for Delphi 10.4 Sydney

26th March 2021 - OpenSSL 1.1.1k Windows binaries released 

There is a new ICS support forum at https://en.delphipraxis.net/forum/37-ics-internet-component-suite/ to replace the old TWSocket mailing that stopped working a few years ago.  

This page contains various components written by Magenta Systems Ltd to extend the François Piette's Internet Component Suite (ICS) version 8 from http://www.overbyte.eu/.  ICS v8 supports Delphi 7, 2006 to 2010, XE to XE8, 10 Seattle, 10.1 Berlin, 10.2 Tokyo, 10.3 Rio and 10.4 Sydney.  Note that all ICS versions now include SSL free of charge.

The latest versions of ICS may be downloaded from the ICS SubVersion server using a subversion client such as TortoiseSVN.  Once your SVN client is installed, you can browse to svn://svn.overbyte.be/ics, svn://svn.magsys.co.uk/ics or https://svn.overbyte.be/svn/ics or https://svn.magsys.co.uk/svn/ics. The SVN login user code is ics and password is ics for read access.

Internet Component Suite (ICS) Downloads

All ICS files should normally be accessed from the ICS Download Wiki page but the important files are available here as well.

Major Changes in ICS V8.66 include:

1 - Added a new TIcsInetAlive component to check for IPv4 and/or IPv6 internet connectivity, using Ping and/or HTTP, defaulting to www.msftconnecttest.com run by Microsoft for Windows 10 alive checking. The online and offline check intervals may be set, and event fires when online state changes. Sample OverbyteIcsHttpRestTst has a demo for new component.

 2 - OpenSSL 1.0.2 and 1.1.0 ceased security fix support over 12 months ago so ICS now only supports 1.1.1, with 3.0 support due in the next few months.This removes a lot of legacy code and functions, and several old ICS functions needed for backward compatibility.

3 - Undertook a major clean-up of OpenSSL functions that may require end user application changes if low level OpenSSL functions have been used, hopefully very rarely. All OpenSSL functions have been renamed to their original names removing ICS f_ prefix for commonality with other Delphi applications.

4 - Bring server SSL/TLS security levels up to latest Mozilla recommendations. In 2021 Mozilla now recommends TLSv1.3 as modern ciphers and TLSv1.2/1.3 as Intermediate supporting all browsers from last five years, so IcsHosts now use sslCiphersMozillaSrvTLS12 as Intermediate level, also Mozilla recommends no cipher server preference so changed that.

5 - Added support for YuOpenSSL which provides OpenSSL in a pre-built DCU statically linked into applications, rather than using external OpenSSL DLLs. This make application distribution more reliable since it can no fail by users deleting the DLLs or copying incompatible versions into the directory. YuOpenSSL is a commercial product from https://www.yunqa.de/ and is supplied as separate compiled DCUs for Delphi 5 to 10.4.

6 - Added a better way of configuring Socks proxy and HTTP Tunnel proxy settings in TWSocket, similarly to that added for THttpCli in V8.62. The new ProxyURL property sets SOCKS or HTTP Tunnel proxy settings using a single URL, ie proto://[user:password@]host:port where proto is socks5 or http.

7 - TIcsRestEmail has new OAAuthType property and event for OAuth2 browser URL perhaps via email for servers. TIcsRestEmail now sets OAuth2 errors, and  clears old tokens.

8 - TIcsHttpMulti has a new ParseLevels property that follow links on a parsed web page to lower level pages, which are also parsed, looking for files to download with a specific file extension, ie .avi.

9 - Updated to OpenSSL 1.1.1k with two high security fixes.

More detailed release notes are at ICS 8.66 Release Notes

Major Changes in ICS V8.65 include:

1 - The ReadMe8.txt file has a new 'Getting Started with ICS' section listing the types of projects ICS may be used for, and suggesting the correct components to use, and their related sample applications for testing. This is recommended reading for anyone doing new ICS development since it discusses all the new high level components like TSslHttpRest added in the last few years which can reduce development effort considerably. It may also be viewed at: ICS_Getting_Started

2 - Added new TIcsRestEmail component that provides basic support for Google and Microsoft Outlook email REST APIs including OAuth2 login and refresh to get an access token for SMTP and POP3 XOAuth2 and 0AuthBearer authentication. TIcsRestEmail has methods to send and read email, to list IDs in a mailbox, read headers and message bodies by ID, send emails and delete emails.

3 - The SMPT, POP3 and MailQueue samples all now support XOAuth2 and 0AuthBearer authentication using the TIcsRestEmail component. The low level component call an event to get the authentication access token, which is provided by IcsRestEmail, together with a refresh token which is saved instead of a password.

4 - To access email using REST APIs or OAuth2/SMTP/POP3 an 'application account' needs to be created though the Google or Microsoft provider console.

5 - Added a new TIcsTwitter component and sample, requires a developer account from Twitter. Includes login to Twitter, send tweet, search tweets and get specific tweets, all responses are Json which the application needs to untangle.

6 - Improved TRestParams allowing them to save more Delphi types correctly without conversion to strings, and to save parameters in new formats.

7 - There are various OAuth2 improvements to make it easier to implement. Added several TOAuthUri records designed to set-up common OAuth2 account settings for providers like Google, Twitter, Microsoft and Sipgate, by using the LoadAuthUri method.

8 - TSimpleWebSrv continues to get less simple, it has aWebSrvIP2 property for a second address so it can listen on IPv4 and IPv6 at the same time, with and without SSL if necessary. Setting WebSrvIP to localhost sets both 127.0.0.1 and [::1] so the browser OAuth2 redirect can choose IPv4 or IPv6.

9 - Made some improvements to SuperObject, used for Json creation and parsing. When parsing Json there are new functions that return a sensible error message about parse errors and the location. There is a new object type DateTime or DT which reads or writes TDateTime to avoid the application needing to do the ISO string conversion. Json can now be parsed to a depth of 64 levels.

10 - Rewrote and improved the way ICS reads SSL/TLS certificates and bundles, simplifying code that has got partly duplicated over the years as new methods were added, and improving error handling so the infamous stack error should no longer appear, instead more useful messages. All certificate files are now written with the UTF8 character set for the added comments that may include non-ASCII characters.

11 - All the ICS root bundles are now created cleanly by an application, rather than mostly manually by copy and editing, to reduce errors. This fixed four corrupted root certificates in the older bundles, see FAQ_SSL/TLS_Certificate_Authority_Root_Stores

12 - Previously the Jose unit offered functions primarily for client JWS/JWT REST applications, it now includes extra functions for REST servers to  check and verify the JWS/JWT sent by clients. The Jose sample application has new tests for JWS/JWT, and to parse and display Json.

13 - Added a new Multi Host FTP Server sample using IcsHosts. Really designed to be a Windows service application. It supports multiple SSL hosts with multiple listeners, can order it's own SSL certificates and will create self signed certificates for any missing, and will email status information and errors to an administrator.

14 - Fixed a long term external SSL session cache issue in some components and samples that meant if an SSL handshake fails due to a bad certificate or chain, it is necessary to remove the SSL session from cache so an immediate retry does not succeed by skipping the certificate checks. This is only a short term issue, because the cache is usually cleared after a few minutes. This will effect any client applications using the external SSL session cache including HTTPS.

15 - Increased the TCP send and receive buffer size to 64K in all components and samples, and generally don't allow it to be set lower. Unfortunately the default buffer size never kept up with faster internet speeds which meant some components transferred data slowly.

16 - Made some improvements ordering SSL/TLS certificates. Made Windows Server DNS updating using WMI more robust so wild card Acme orders work reliably.

17 - Made some internal changes loading OpenSSL, to avoid the two DLLs being loaded from different directories and to give better exceptions if they are missing.

More detailed release notes are at: ICS_V8.65

Major Changes in ICS V8.64 include:

1 - ICS now supports International Domain Names for Applications (IDNA), i.e. using accents and Unicode characters in domain names. Domain names can only contain lowercase ASCII letters and numbers and a couple of symbols, so Unicode U-Labels (nodes in a domain) must be converted to A-Labels (Punycode ASCII) with an ACE (ASCII Compatible Encoding) prefix. So www.mâgsÿstést.eu becomes www.xn--mgsstst-pwa1e4l.eu and мособлеирц.рф becomes xn--90aijkdmaud0d.xn--p1ai.

2 - Added a new WMI component based on the Magenta Systems WMI and SMART Component which allows listing all WMI classes, adding and updating network adaptor IP addresses, and adding and updating records in Windows DNS Server (2012 and later).

3 - For SSL/TLS servers, the initial client hello sent before protocol and cipher negotiation starts is now available in the onSslServerName event, when it is easier to select the SslContext to be used on shared servers supporting lots of hosts.

4 - There are various improvement for SSL/TLS servers using IcsHosts. NonSSlPort may be zero for to allocate a random port (not for SSL), the new BindPort and BindSrvPort properties show the real port while listening, reported by ListenStates. If SslCert can not be found the server will create a self signed SSL certificate with all the HostNames so the server can start.

5 - There are many X509 SSL certificate ordering improvements from Let's Encrypt using ACME. All the ACME challenges are now supported. tls-alpn-01 which works on SSL port 443 using a special self signed certificate thus avoiding a server running on port 80 as well. dns-01 which updates Domain Name Server TXT records to avoid using any servers, allowing wild card orders, currently Windows DNS Server on the same server, but Cloudfare is planned.

6 - TRestParams used with the TSslHttpRest component, has more parameter content types: PContXML, PContBodyUrlEn, PContBodyJson and PContBodyXML. The existing PContUrlEn and PContJson now specify REST params are sent as URL ? arguments, while the PContBodyxx version send params as content body. XLM parsing into an ISuperObject has been added so SOAP and XML REST responses can be processed similarly to Json responses.

7 - The TFtpServer component has a new TFtpOptions ftpsAuthForceSsl which requires SSL/TLS before allowing the LOGIN to be processed so that clear text credentials can not be sent. May also be set using IcsHosts with AuthForceSsl=True for specific Hosts only. Failure gives '533 USER requires a secure connection'.

8 - Added a Websocket server implementation (HIXIE and HYBIE protocols) ported from phpws project, written by Stan Korotky <stasson@orc.ru>. There is no a specific component for websockets, they are handled internally from a custom TWebSockSrvClient.

9 - Did a refresh of the three SSL/TLS trusted root certificate bundles included with ICS, to add new roots and remove untrusted certificates.

10 - Added CreateSelfSignCertEx to OverbyteIcsSslX509Utils to create self signed certificates with subject alternate names and specific key types.

More detailed release notes are at: http://wiki.overbyte.eu/wiki/index.php/ICS_V8.64

Major Changes in ICS V8.63 include:

1 - IcsHosts is now supported in all the main ICS server components, TWSocketServer, TSslHttpServer, TSslHttpAppSrv, TIcsProxy, TIcsHttpProxy, TIcsIpStrmLog and TSslFtpServer. This simplifies server applications which do not need to setup an SSL context, can easily support multiple listeners and hosts, and can automatically order and install SSL/TLS certificates from Let's Encrypt and commercial suppliers. # There are several changes relating to automatic certificate ordering, mostly cosmetic based on experience adding the feature into more applications with a few issues fixed during testing with better logging.

2 - Made improvements to handle the OAuth2 version used by Google Accounts, allowing the REST component to access Google APIs such as Gmail.

3 - Did a major refresh of the three SSL/TLS trusted root certificate bundles included with ICS, to add new roots from Amazon and others, and remove untrusted certificates.

4 - There are improvements in TFtpCli to access FTP servers behind NAT routers where the internal and external IP addresses are different and not correctly handled by the FTP server.

5 - The TSslFtpServer component now uses the IcsHosts concept added to the web and proxy servers two years ago, see above. There is a new IcsHosts property which allows multiple hosts to be specified. Also fixed ftpsNoPasvIpAddrInLan and ftpsNoPasvIpAddrSameSubnet options to work correctly to present local passive IP address on LANs.

6 - The TIcsIpStrmLog component will now start despite non-fatal SSL/TLS certificate warnings, and may be used with self signed certificates. The OverbyteIcsIpStmLogTst sample shows how to restart the TCP server after the first SSL/TLS certificate is automatically installed.

Major Changes in ICS V8.62 include:

1 - The HTTP client has new properties. There is a new THttpAuthType of httpAuthJWT which uses the AuthBearerToken property for Json Web Token authentication. A new ProxyURL property combines four proxy properties as a URL for simplicity, ie http://[user[:password]@]host:port. The new AlpnProtocols property is a request list sent when an SSL connection starts, once connected GetAlpnProtocol returns which protocol the server wants to uses.

2 - Various improvements for automatic X509 SSL certificate ordering to support final ACME RFC8555. AcmeV2 now supports POST-as-GET, GET alone being removed later in 2019. Added Proxy URL support, might be needed for servers behind NAT firewalls for public access. Added support tls-alpn-01 challenge for local web server on 443, but not working yet.

3 - SocketServer now uses a separate local web server for servers not using ports 80 or 443 such as FTP, SMTP, proxies, etc, When ordering X509 SSL certificate using ChallFileSrv challenge.

4 - TWsocket now raises a background exception for user exceptions in the OnDataAvailable event rather than silently ignoring them. If IcsLogger is being used, HandleBackGroundException now logs exceptions and their source to make them easier to find.

5 - In the TSslHttpRest component, TRestParams can add Json parameters as PContJson which means arrays and nested Json can be added. Added a new SslAllowSelfSign property to connect OK to sites with self signed SSL certificates which would normally fail validation.

6 - TSimpleWebSrv now supports SSL, with certificate bundle and host name, and supports the SSL ALPN extension for automatic X509 SSL certificate ordering.

7 - The new TIcsSms component adds support for SMS Works bureau at https://thesmsworks.co.uk/ to send SMS messages, cheaper than Kapow, and allows sender ID to be freely changed.

8 - Updated all .dproj files for all samples to add the ICS source folder to the search path of each project. 

9 - Added Time Zone support for date string conversions, to UTC time with a time zone, and back to local time using a time zone, primarily for SMTP email headers which otherwise show UTC time.
 
10 - TIcsIpStrmLog using TCP server now uses the root bundle correctly and reports the certificate chain and bindings. Ensure all listeners started for TCP Server, if more than one.

Major Changes in ICS V8.61 include:

1 - Added two new components using the new HTTPS REST component, which are both useful and illustrate how simply they can created, TIcsSms and TDnsQueryHttps, both in the OverbyteIcsSslHttpRest.pas unit with demos in OverbyteIcsHttpRestTst.

2 - The new TIcsSms component sends SMS text messages via an HTTP bureau, you will need an account. Initially supporting https://www.kapow.co.uk/.
.
3 - The new TDnsQueryHttps component makes DNS queries over HTTPS (DOH), to ensure integrity and privacy from interception by ISPs or proxies. The original TDnsQuery component has also been updated to support all the common queries and return them in using a single AnswerRecord array.

4 - Improved HTTP client and server NTLM authentication by adding Single Sign On with NTLM Session on Windows Domain to get credentials without needing them specified in code.

5 - Various improvements in the HTTPS REST, HTTP client and FTP client.

6 - Fixed SSL certificate ValidateCertChain to check certificate start and expiry dates in UTC time instead of local time.

7 - The Browser Demo sample using HtmlViewer now correctly supports authentication methods where a site requires a login, and has an improved log window that no longer slows down display of complex pages.

Major changes in ICS V8.60 include:

1 - Added several new components and sample applications created by Magenta Systems Ltd over the past 17 years and previously distributed and installed separately to the ICS distribution. Bundling them with ICS makes installation and updating easier, and allows existing ICS samples to make use of many the new components, such as UTF-8 file logging. All the components have new names so existing applications using the originally distributed versions will still work, but it's recommended updating existing applications for the new ICS versions. The added components include IP stream logging, SMTP Mail Queue, Time Server and Client, Whois client, blacklisting of malicious IP addresses, file logging functions, file indexing, copying and deleting, FTP indexing, multiple file uploading and downloading, HTTP page parsing and URL downloading.

2 - In THttpCli, only follow relocation for 3xx response codes, not 201 Created, but keep Location property for 201 which is often response to a POST and may be needed by the application.

3 - When starting TWSocket Connect, the IP address chosen for DNS lookup is now saved in ASCII as AddrResolvedStr which is useful in connect OK or failed events to see whether an IPV4 or IPv6 address was chosen, and which was used if DNS offered multiple IPs. AddrResolvedStr is exposed as a property in TWSocket, THttpCli, TSslHttpRest and TFFtpClient and reports in failed connection events. Other clients will be added soon.

4 - Various samples have been updated to ease testing of IPv6 and to save the diagnostic window activity to a disk log file, OverbyteIcsHttpsTst, OverbyteIcsHttpRestTst, OverbyteIcsTimeTst, OverbyteIcsX509CertsTst.

5 - Added round robin DNS lookup if DNSLookup returns multiple IP addresses, so they are used in turn after a failure when a component is called repeatedly without being freed. This is implemented in THttpCli, TSslHttpRest TFFtpClient and TIcsTimeClient. Other clients will be added soon. There is a new OnSelectDns event to override round robin lookup and make your own choice.

By default, the DNS lookup in ICS components ignores IPv6 addresses and always use the first IPv4 address offered, when there is more than one. This is usually implemented in the OnDnsLookupDone event in the application or high level component. So if that first address does not respond, the application never tries any other addresses.

This has become more of a problem when enabling applications for IPv6, by changing SocketFamily from the default sfIPv4, to sfAny, sfAnyIPv4 (prefer IPv4), sfAnyIPv6 (prefer IPV6) or sfIPv6 (only IPv6), when IPv6 addresses may also be returned as well as IPv4. Due to routing or firewall issues, IPv4 and IPv6 might not both be available and so connection will fail if that address is chosen first. Previously it was necessary to restrict SocketFamily so only the working family is attempted.

The DNS round robin implementation relies on keeping the last successful connected IP address, so it can be re-used for  subsequent connections, but looping through any alternative addresses if the last connection failed, for subsequent connection attempts. Most existing applications use the OnDnsDone event to select the connection IP address so the round robin code is added there.

Newer applications making use of TWSocket ComponentOptions wsoAsyncDnsLookup should added code in OnDNSLookupDone and update the DnsResult property which is then used by Connect.

6 - IcsHosts supports two new TSslSrvSecurity server security levels, sslSrvSecTls12Less and sslSrvSecTls13Only, the former disables TLS1.3 in ICS servers if TLSv1.3 fails (perhaps a bad client implementation) while the second only supports TLSv1.3.

7 - Up to date C++ packages are included for 10.2 Tokyo and 10.3 Rio. Information on installing ICS for C++ 2007 may be found at: https://en.delphipraxis.net/topic/844-use-ics-with-c-builder-2007/  Sorry, currently don't believe it's easy to update the ICS source code to avoid the changes needed for C++ 2007.

Detailed V8.62 Release Notes
Detailed V8.61 Release Notes
Detailed V8.60 Release Notes
Detailed V8.58 Release Notes
Detailed V8.50 Release Notes

Nightly ICS v8 for Delphi 7-2010, XE-XE8, 10 Seattle, 10,1 Berlin, 10.2 Tokyo, 10.3 Rio and 10.4 Sydney
v8 - 23 Jul 2021 (12,155,890 bytes)

OpenSSL Binaries 1.1.1 Win32
1.1.1k 32-bit - 26 Mar 2021 (1,820,157 bytes)
Only supports Windows Vista/Server 2008, and later, not Windows XP.  The OpenSSL DLLs and EXE files are digitally code signed 'Open Source Developer, François PIETTE', the lead developer for ICS. ICS V8.38 and later check the DLLs are correctly signed when opening them. Beware that Windows needs recent root certificates to check newly signed code,  and may give an error if the root store has not been kept current by Windows Update, particularly on older versions of Windows such as  Vista, 2008 and 7.

OpenSSL Binaries 1.0.2 Win32
1.0.2u 32-bit - 7 Jan 2021 (1,742,205 bytes)
Note ICS V8.65 is the last release to support OpenSSL 1.0.2, whose support ceased in 2019.

Sources of CA Trusted Stores

SSL/TLS trusted root certificate bundles and always changing, annually perhaps for major changes, although Microsoft officially Windows roots every two months. The Common CA Database (CCADB) https://www.ccadb.org/ is a repository of information about Certificate Authorities (CAs), and is used by a number of different root store operators to manage their root stores.

But it's not easy to create root bundles from CCADB and another developer got frustrated with updating roots, and created a Trust Stores Observatory Git repository: https://github.com/nabla-c0d3/trust_stores_observatory which contains over 500 root certificates and lists of which trust store contain which roots by different operating systems. But even this does not contain certificates in a form easily used by OpenSSL, so Magenta Systems Ltd has written a small tool that converts the YAML files from TSO into PEM bundle files, one each for the different operating systems.

New PEM Bundle CA Trusted Store Files
CA PEM Bundle - 6 Jul 2021 (1,445,384 bytes)

There are six different PEM CA bundle files, built from the Trust Stores Observatory Git repository:

apple.pem - 164 Certificates
google_aosp.pem - 137 Certificates
microsoft_windows.pem - 259 Certificates
mozilla_nss.pem - 128 Certificates
openjdk.pem - 92 Certificates
oracle_java.pem - 93 Certificates

Each certificate is prefixed by it's description, issuer fields, expiry, public key type and SHA256 hash, so the bundles are self documenting rather than being just cryptic base64 blocks. These PEM bundles may be loaded into an OpenSSL context as a root store.  Magenta Systems Ltd will periodically update these bundles, as needed. The files are all UTF-8 with a BOM. While the certificates are base64 encoded, the aded comments may include Unicode characters for non-English issuers.

The zip file contains two versions of each bundle, the name above and one ending with -clean.pem which omits all the added textual comments so is smaller and less likely to cause problems with non-English characters.  There are also -titles.txt and fprints.txt files which are one line per certificate listing the main details, and fingerprint in the latter file. 

Note the ICS distribution download (see above) contains three CA Trusted Stores, two as PEM bundle files, one in a source unit, and access to the Window Certificate Store directly, see FAQ_SSL/TLS_Certificate_Authority_Root_Stores for more information.

Now part of ICS V8.60 and later, as TIcsMailQueue

Magenta Systems Mail Queue Component
Version 2.5 - 26 Nov 2018 (2,414,770 bytes)

Magenta Systems Mail Queue Component has two main benefits over a simple TSslSmtpCli component: it supports extended retries over many hours or days, and supports multiple SMTP relay servers or looks-up MX servers, while alleviating the need for the application to handle retries. The component also allows HTML mail to be sent using SSL, something THtmlSmtpCli does not currently support. Mail is queued to disk, so retries will continue if the application is restarted.

TMagMailQueue is designed to prepare, queue and send email. Preparing the email is done using the ICS THtmlSmtpCli component so it may be plain text or HTML email with one or more file attachments.

Once the mail properties in QuHtmlSmtp have been specified, it is queued using the QueueMail method which saves it to an EML spool file.

The component runs a thread which checks the queue for new EML spool files, and attempts to forward them to one or more SMTP Mail Servers using TSslSmtpCli, optionally with SSL. If mail delivery succeeds, the spool file may be deleted or moved to an archive folder. If mail delivery fails, the spool file remains in the queue and further attempts are made separated by the times in minutes listed in the RetryList list. If all delivery attempts fail, the spool file may be deleted or moved to a badmail folder.

Note that some email servers support grey listing and reject the first email attempt from a new sender but allow a retry 10 or 15 minutes later, something that is very effective in blocking spam emails (since they don't usually retry).

If multiple mail servers are specified, delivery is attempted once using each server, for each retry attempt. Each mail server is specified as TMailServer and there is no limit to the total.

Each time the queue is updated or a delivery attempt made, the queue is saved to file in the control folder, so the component may be stopped and restarted with failed attempts continuing.

The EML spool files are compatible with those created by many Microsoft email applications such as CDO, and the AddtoQueue method can also be used to queue existing EML files with the queue details specified in MailQuItem.

Note, this component is intended for sending low volume email from individual Delphi applications, with more flexibility than a simple TSslSmtpCli component. For use as a heavy duty SMTP server, queue processing could be improved to avoid moving records around as much or saving them to disk as often, and mail bodies could be read as required from disk instead of being read entirely to memory first.  A mail pickup folder could be added which is scanned for new EML files.

Files and Folders Used

The TMagMailQueue component heavily uses disk files, in different sub-directories within the mail root directory specified in property MailQuDir, these are:

• control - contains MailQuItems.Ctl a single row file with the next message item number, and MailQuItems.Hdr which is a CSV file containing one row for each mail item still in the queue.
• spool - contains any queued email files, named in the format item00000001.eml with the number increasing, taken from MailQuItems.Ctl
• archive - if ArchiveSent property is true, once an email has been successfully sent it is moved into the archive directory
• badmail - if DeleteFailed property is false, once an email has exceeded all the retry attempts it is moved into the badmail directory, from where it may be manually requeued if necessary

If logging of sent email is specified, the default file name FileQuSent property is MailQuSent-yyyymmdd.log inb CSV format similar to MailQuItems.Hdr.

A demo application mailqudemo.exe illustrates simple email queuing.  The zip contains the EXE demo and required SSL files.

Release Notes

18th January 2011 - 1.0 - first public release. Not yet tested with Delphi 2009 or later.

2nd March 2011 - 1.2 - automatically create mailqueue directory in demo application, removed missing uses statement. Support queuing mail with OwnHeaders bypassing htmlmail. Log event definition changed.

11th August 2011 - 1.2 - updates subroutines for Win64 support, removed one unneeded unit from uses.

5th Oct 2011 - 1.3 - Debug logging works properly Don't retry emails that fail too large for server (error 552)

11th Sept 2012 - 1.4 - ICS V8, IPv6

23rd March 2013 - 1.5 - Added Mail Server SocketFamily and LocalAddr6 for IPv6

10th Dec 2014 - 1.6 - Better SSL handshake reporting

27th Oct 2015 - 2.0 - requires ICS V8.19 October 2015 or later.
Check and report SSL certificates using PEM file or Windows Cert Store
Allow three SMTP servers to be specified for each email in queue
Lookup DNS MX records and send to those SMTP servers
Queue keeps last response or error in queue
Mail completed log (same CSV format as queue)
Queue changed event to tell client something is happening
QueueMail method now returns item number (not boolean)
New UnQueueMail method to remove item number from queue
Demo save settings in INI file
Demo new View Mail Queue window to see what's waiting
Added SMTP Send Method, relay, specific or lookup MX mail servers
Added HELO Sending Host Name may be needed if using MX mail servers

Warning - if using MX DNS servers and multiple recipients, need to queue mail multiple times !!!! This will be fixed real soon.

7th July 2016 - 2.1 - requires ICS V8.30 July 2016 or later.
Support SSL enhancements in ICS for OpenSSL 1.1.0
Don't change SSL directory, let application control it
Use default SSL root bundle if none specified

1st December 2016 - 2.2 - requires ICS V8.39 November 2016 or later.
Better error handling.
Use OpenSSL host checking.
Fixed bug that meant failed email was not deleted from queue.
Don't queue email without recipients.
Use timer to update windows to avoid problems with mass email performance.

6th March 2017 - 2.3 - requires ICS V8.43 March 2017 or later.
Simplified SSL certificate reporting.

11 Mar 2017 - 2-4 - Added WaitSend to wait until everything sent.

22th Jun 2018 - 2.5 - requires ICS V8.55 20 June 2018 or later.
Added RetryWithoutSsl which retries an SSL failure without SSL.
Added SslCliSecurity to set client security level.
Using IcsWndControl for threaded message handling.
SendSmtpClient now created new for each attempt in case of prior faillure causing terminal corruption.
If SSL certificate verify fails, next attempt is another server.
Supports TLSv1.3 with OpenSSL 1.1.1.

26th November 2018 - 2.5 - tested with ICS 8.58
Added final OpenSSL 1.1.1a DLLs, recompiled.

Now part of ICS V8.60 and later. 

Now part of ICS V8.60 and later, as TIcsIpStrmLog

Magenta Systems IP Log Streaming Component
Version 2.8 - 14 Dec 2018 (4,156,649 bytes)

TMagIpLog is designed for IP stream logging, using TCP Client, TCP Server, UDP Client or UDP Server protocols, sending simple text lines across a network so they may be displayed or written to disk remotely. The component allows two way communication with TCP and UDP, so may also be used for simple protocols such as communication between two applications. The component supports multiple client sockets so may be used to send data to two or more different remote servers at the same time.

 For TCP and UDP clients, the component will optionally ping the remote computer first before opening an IP connection to allow faster failure retries and some confirmation that UDP may work. TCP client provides repeated connection retry attempts, including re-establishing a lost connection. UDP client will optionally keep pinging the remote during a connection to ensure it's still there. UDP server sends data to the IP address and port from which it last received data. TCP server supports multiple remote clients connecting. Received data is parsed for various line endings optionally removing control characters and triggering an event for a received line. The only other two events are optional, one for state changed when starting and stopping, the second offering progress information and errors. 

The component supports both IPv4 and IPv6, host name lookup for TCP and UDP Client, and SSL connections for TCP Client and TCP Server, including remote server certificate checking using either a local PEM bundle root file or the Windows Certificate Store.

A demo application testiplog.exe illustrates use of TMagIpLog as a TCP or UDP client or server, and both in the same program sending data locally. The same component may be used in a client or server application, to send or receive.

The Magenta Systems ComCap application may also be used to capture IP streams to files or a database.

Using TMagIpLog:

1 - Drop the component onto a form (or create it in code, see testiplog.exe).

2 - Specify LogProtocol as one of logprotUdpClient, logprotUdpServer, logprotTcpServer, logprotTcpClient.

3 - For client protocols, specify RemoteHost (name or IP address) and RemoteIpPort, CheckPing true if ping to be used, RetryAttempts to non-zero if continual retries not needed, RetryWaitSecs for delay between retries .

4 - For server protocols, LocalIpAddress is 0.0.0.0 to listen on all local addresses, LocalIpPort must be non-zero.

5 - For sending data, AddCRLF to false if line already have terminating characters, UdpNoCRLF to false if UDP should send CRLF.

6 - For receiving data, LineEndType to one of lineendCR, lineendLF, lineendCustom (set in hex in CustomLineEnd) or lineendPacket (for UDP), then MaxLineLen if a line should be returned before lineend is found, normally non-ASCII characters are removed, set StripControls to false if they should be replaced by spaces, RawData to true if CR, LF, FF and control characters should not be removed.

7 - Assign onLogRecvEvent if data is to be received, onLogChangeEvent if tracking of start and stop is needed, onLogProgEvent if progress information is needed for logging.

8 - Call StartLogging. The LogChangeEvent and LogProgEvent will trigger when LogState changes to logstateOK when data may be sent.

9 - To send a line, if function GetAnyStateOK is true, call SendLogLine. MaxSendBuffer specifies the amount of data that can be buffered otherwise SendLogLine will fail.

10 - Received data will trigger LogRecvEvent once per line.

11 - Call StopLogging to stop. Buffered data may continue to be sent after close, keep calling CheckStopped until true when it's really finished and component may be destroyed.

12 - To send an unlimited size stream, create a stream in the application with TBufferedFileStream or TFileStream, and pass it to SendStream.  LogState changes to logstateOKStream while it's being sent, then back to logstateOK as it finishes, the application should then free the stream.

13 - There is no specific handling for receiving a stream, textual data will be handled according to the normal line end properties, and can be saved to another stream in LogRecvEvent. Binary data is more problematic, set RawData to true and MaxLineLen to get a buffer load at a time, but the last buffer load will need to be extracted with GetPartialLine using a timeout, this is called automatically when the connection is closed.

14 - To send to multiple clients, set MaxSockets to the number needed, then use the function SetRemotes to specify the remote host and port for each socket number, base 0. The events all return Socnr to indicate which socket. MaxSockets also specifies how many remote clients can connect to TCP Server, but note that Socnr is dynamic and changes as remote clients come and go.

15 - To support SSL on TCP/IP client or server, drop an TSslContext component on the form, assign it to the LogSslContext property and set the ForceSsl property to true. For better performance, set LogSslSessCache to a TSslAvlSessionCache component.

16 - For SSL TCP Server, the SslContext component must have the SslCertFile and SslPrivKeyFile properties set to the file names of an SSL certificate and Private Key PEM files respectively, and SslCipherList set to sCipherMozillaSrvBack for strong but backward compatiblle cipher support. The component includes sample self signed certificate and password files iplog-cert.pem and iplog-prvkey.pem, and you can create your own with the ICS SSL sample application Pemtool, or buy commercial PEM certificates.

17 - For SSL TCP Client, the SslContext component must have the SslCAFile property set to the file name of a PEM root certification authority file containing trusted root certificates. Such a file is supplied with the component RootCaCertsBundle.pem containing various root certificates covering most major registries. SslContext SslCipherList can be left as the default to allow connection to any server. The LogSslVerMethod property can be logSslVerNone to skip certificate verification, logSslVerBundle to check using the CA bundle file or logSslVerWinStore to check using the Windows certificate store (a little slower, bur maybe more certificates). To check if certificates have been revoked set LogSslRevocation to true, beware this needs public internet access and can be very slow or fail. LogSslReportChain set to true reports certificate details checked.

Release Notes

18th August 2007 - 1.1 - using OverbyteIcsFtpSrvT instead of OverbyteIcsLibrary, UDP receive packets may be from multiple hosts, always keep IP.

5th August 2008 - 1.2 - made compatible with ICS V7 and Delphi 2009. Note only supports ANSI with Delphi 2009.

20th August 2009 - 1.3 - fixed problem with MaxSockets being reported as closed in the event when only one was open, tested with Delphi 2010.

9th August 2010 - 1.4 - removed cast warnings with Delphi 2009 and later

22nd Sept 2011 - 1.5 - added SndBufSize and RcvBufSize to increase buffer sizes and speed

11th Sept 2012 - 1.6 - better error for too many clients with server added CurSockets property for current number of server sockets

7th July 2014 - 2.0 - now only ICS 8 and later, using new ICS ping. 
Added IPv6 and SSL support, including server certificate checking.
Added host name support for UDP and TCP client with DNS lookup.
Added LogProtocols suffixed 6 for IPv6.
Cleaned up some progress messages, identify error progress events.
Removed line length limit of 1024 that was not checked.
Added send a stream of unlimited length.
Get buffered partial received line during close.
Default line end is LF instead of CR so UNIX files are processed.

13th July 2015 - 2.2 - requires ICS V8.18 June 2015 or later.
Added better SSL handshake error reporting.
Added lineendCRLF, only support FF as lineend if using CR.
Added Debug Info button for ICS info level logging.
Added SSL Server DH Params, set ECDHCurves, both for ECDH ciphers.
Note OpenSSL no longer support dhparam512, minimum is 768 bits.

23rd Oct 2015 - 2.3 - requires ICS V8.19 October 2015 or later.
Better SSL client and server certificate reporting.

8th July 2016 - 2.4 - requires ICS V8.30 July 2016 or later.
Fixed certificate reporting typo.
Removed TBufferedFileStream, not needed.
Added SrvTimeoutSecs to close idle server sessions, note needs ICS V8.30  or later to fix a SSL bug that stopped SrvTimeoutSecs working.
Added Socket property to get current socket, mainly for statistics
Report session length and data xmit/recv before closing

23rd Nov 2016 - 2.5 - requires ICS V8.39 November 2016 or later.
Added GetSendWaiting to check how many bytes of send data not yet sent.
Increased default MaxSendBuffer size to 64K.
Added property TotRecvData total data received since connection, or when method ResetRecvData was called.
Added property MaxRecvData which causes onLogRecvEvent to be called when that length has been received. May be used for fixed length binary packets or where received data contains a content length such as a HTTP response header followed by binary data.
Server takes exclusive access of addr/port.
Fixed bug with multiple clients not using correct port.
Added SSL Server Name Indication support.
Check multiple client SSL host names correctly.
Removed USE_SSL so SSL is always supported.
Removed TX509Ex now using TX509Base.
Using OpenSSL certificate verification host checking.
Server now supports LogSslReportChain to report server certificates, checks expired and reports chain.

7th March 2017 - 2.6 - requires ICS V8.43 March 2017 or later.
set IcsLogger for context so it logs more stuff.
Simplified reporting SSL certs in client handshake.
Improved validation of server certificates.
Use threaded DNS lookup.

22nd June 2018 - 2.7 - requires ICS V8.55 20 June 2018 or later.
Support TLSv1.3, no real changes.
Don't start SSL handshake twice.
Cleaned up SSL error handling.
Added SslCliSecurity to set client security.

14th December 2018 - 2.8 - tested with ICS 8.58
Added final OpenSSL 1.1.1a DLLs, recompiled.
Removed madexcept.

Pending major changes to use IcsHosts in 3.0.

Now part of ICS V8.60 and later.  Uses IcsHosts.

Now part of ICS V8.60 and later, as TIcsWhoisCli with a new sample application

Whois Component and Demo
Version 1.0 - 2 Nov 2005 (254,079 bytes)

A Whois component and demonstration application.  Whois is a protocol to interpret a remote server for information about a domain name or an IP address, and return textual information about 'owner' of the name or address.  The demo application interprets the result and will perform a secondary query to another Whois server if necessary.

Now part of ICS V8.60, as TIcsTimeClient and TIcsTimeServer with a new sample application

SNTP Time Server and Client Components
Version 1.0 - 9 Mar 2006 (11,405 bytes)

TTimeServ is an updated version of Nathan Anderson's time server component adding SNTP support.  TWSTimeClient is an updated version of Chris Barber's time client component adding SNTP support and functions to change the PC UTC time.  SNTP provides time correction with fractional seconds, unlike the earlier Time protocol that is round seconds only.

Magenta Systems Ltd, 9 Vincent Road, Croydon CR0 6ED, United Kingdom
Phone 020 8656 3636, International Phone +44 20 8656 3636
https://www.magsys.co.uk/
Copyright © 2021 Magenta Systems Ltd, England. All Rights Reserved.