Delphi Developers
Components for Internet Component Suite (ICS)

25th October 2024 - OpenSSL 3.4.0 Windows binaries released.

26th September 2024 - ICS V9.3 released, many improvements

5th September 2024 - OpenSSL 3.3.2,  3.2.3 and 3.0.15 Windows binaries released.

16th February 2023 - Updated PEM Bundle CA Trusted Store Files

August 2023 - Magenta Systems Internet Protocol Helper Component and Magenta Systems Internet Packet Monitoring Components are now part of ICS v9, with major upgrades and new samples, with full IPv6 support.

There is a new ICS support forum at https://en.delphipraxis.net/forum/37-ics-internet-component-suite/ to replace the old TWSocket mailing that stopped working a few years ago.  

This page contains various components written by Magenta Systems Ltd to extend the François Piette's Internet Component Suite (ICS) version 8 from http://www.overbyte.eu/.  ICS v8 supports Delphi 7, 2006 to 2010, XE to XE8, 10 Seattle, 10.1 Berlin, 10.2 Tokyo, 10.3 Rio,10.4 Sydney, 11 and 12.  Note that all ICS versions now include SSL free of charge.

Internet Component Suite (ICS) Downloads

All ICS files should normally be accessed from the ICS Download Wiki page but the important files are available here as well.

The latest versions of ICS may be downloaded from the ICS SubVersion server using a subversion client such as TortoiseSVN.  Once your SVN client is installed, you can browse to svn://svn.overbyte.be/ics, svn://svn.magsys.co.uk/ics or https://svn.overbyte.be/svn/ics or https://svn.magsys.co.uk/svn/ics. The SVN login user code is ics and password is ics for read access.

Please note that ICS does not use any of the new features in OpenSSL 3.1, 3.2 or 3.3 at present, so continuing to use the latest patch version of 3.0 with security fixes is generally fine.

Nightly ICS v9 for Delphi 7-2010, XE-XE8, 10 Seattle, 10,1 Berlin, 10.2 Tokyo, 10.3 Rio, 10.4 Sydney, 11 and 12
v9 - 20 Nov 2024 (37,216,142 bytes)

Latest ICS V9.3 Release
V9.3 - 27 Sep 2024 (31,555,657 bytes)

OpenSSL Binaries 3.4 Win32
3.4.0 32-bit - 25 Oct 2024 (5,101,237 bytes)
Minor new version of OpenSSL, requires minimum ICS V8.67.
Note the binaries are now digitally signed by 'Magenta Systems Ltd'. Supported until October 2026.

OpenSSL Binaries 3.3 Win32
3.3.2 32-bit - 4 Sep 2024 (5,036,026 bytes)
Minor new version of OpenSSL, requires minimum ICS V8.67. Supported until April 2026.

OpenSSL Binaries 3.2 Win32
3.2.3 32-bit - 4 Sep 2024 (4,979,273 bytes)
Minor new version of OpenSSL, requires minimum ICS V8.67. Supported until November 2025.

OpenSSL Binaries 3.0 Win32
3.0.15 32-bit - 4 Sep 2024 (4,572,653 bytes)
Major new version of OpenSSL, requires minimum ICS V8.67. Long term support version, until September 2026.

OpenSSL Binaries 1.1.1 Win32
1.1.1w 32-bit - 20 Sep 2023 (1,829,214 bytes)
Only supports Windows Vista/Server 2008, and later, not Windows XP, requires minimum ICS V8.57. Support ceased in September 2023 unless you have an OpenSSL Premium Level Support contract for $50,000/year. Not supported by ICS V9.1 or later.  

OpenSSL Binaries 1.0.2 Win32
1.0.2u 32-bit - 7 Jan 2021 (1,742,205 bytes)
Note ICS V8.65 is the last release to support OpenSSL 1.0.2, whose support ceased in 2019 unless you have an OpenSSL Premium Level Support contract for $50,000/year. Not supported by ICS V8.66 or later.  

Sources of CA Trusted Stores

SSL/TLS trusted root certificate bundles and always changing, annually perhaps for major changes, although Microsoft officially Windows roots every two months. The Common CA Database (CCADB) https://www.ccadb.org/ is a repository of information about Certificate Authorities (CAs), and is used by a number of different root store operators to manage their root stores.

But it's not easy to create root bundles from CCADB and another developer got frustrated with updating roots, and created a Trust Stores Observatory Git repository: https://github.com/nabla-c0d3/trust_stores_observatory which contains over 680 root certificates and lists of which trust store contain which roots by different operating systems. But even this does not contain certificates in a form easily used by OpenSSL, so Magenta Systems Ltd has written a small tool that converts the YAML files from TSO into PEM and PKCS7/12 bundle files, one each for the different operating systems.

New PEM Bundle CA Trusted Store Files

CA PEM Bundle - 16 Feb 2024 (3,055,578 bytes)

There are six different PEM CA bundle files, built from the Trust Stores Observatory Git repository in February 2024

apple.pem - 169 Certificates
google_aosp.pem - 134 Certificates
microsoft_windows.pem - 282 Certificates
mozilla_nss.pem - 147 Certificates
openjdk.pem - 89 Certificates
oracle_java.pem - 93 Certificates

Each certificate is prefixed by it's description, issuer fields, expiry, public key type and SHA256 hash, so the bundles are self documenting rather than being just cryptic base64 blocks. These PEM bundles may be loaded into an OpenSSL context as a root store.  Magenta Systems Ltd will periodically update these bundles, as needed. The files are all UTF-8 with a BOM. While the certificates are base64 encoded, the added comments may include Unicode characters for non-English issuers.

The zip file contains three versions of each bundle, the name above, one ending with -clean.pem which omits all the added textual comments so is smaller and less likely to cause problems with non-English characters, and a third PKCS7/12 version with extension P12 which is smaller than PEMs.  There are also -titles.txt and -fprints.txt files which are one line per certificate listing the main details, and fingerprint in the latter file.  There are also changes files for the Microsoft Windows bundle that indicates which certificates were removed or added with each update.

Note the ICS distribution download (see above) contains three CA Trusted Stores, two as PEM bundle files, one in a source unit, and access to the Window Certificate Store directly, see FAQ_SSL/TLS_Certificate_Authority_Root_Stores for more information.

Internet Component Suite (ICS) Release Notes

Changes in ICS V9.3 include:

1 - V9.3 continues the simplification of use of ICS components by consolidating many types and constants into the OverbyteIcsTypes unit, avoiding projects needing to find and add specific units before they will build. For XE2 and later, OverbyteIcsTypes and OverbyteIcsSslBase will be added automatically when components needing them are dropped on a form, or that form accessed for existing projects. One benefit of this change is removing dependence on several units for many components and applications, it should be possible to remove OverbyteIcsWinsock, OverbyteIcsLIBEAY, OverbyteIcsSSLEAY and OverbyteIcsLogger from most applications, and also other units. See https://wiki.overbyte.eu/wiki/index.php?title=Updating_projects_to_V9.3 for more information.

2 - Previously, the Windows Certificate Store was supported on Windows for all components and samples, despite it not always being required. There are three new defines {$DEFINE MSCRYPT_Clients}, {$DEFINE MSCRYPT_Servers} and {$DEFINE MSCRYPT_Tools) that determine which components can use the store, at least one must be set or applications that need the store will fail. Although these new defines all default to enabled in the OverbyteIcsDefs.inc supplied with V9.3 and later, unless this file is installed, Windows Certificate Store will be unavailable. These defines are disabled for non-Windows platforms and for C++ Builder which has bugs.

3 - Added new application independent monitoring, comprising a client component and server sample. The ICS Application Monitor TIcsAppMonCli client component is designed to report to an ICS Application Monitor server, which will ensure the main application remains running. The ICS Application Monitor server IcsAppMon.exe is designed to monitor ICS applications using the TIcsAppMonCli client component, and ensure they remain running, restarting the application if it stops or becomes non-responsive, or on demand. Primarily to keep ICS server Windows services running non-stop, but may also be used for network wide monitoring of ICS applications. Client and server both use the TIcsIpStrmLog component with a simple TCP protocol. More information at https://wiki.overbyte.eu/wiki/index.php?title=FAQ_ICS_Application_Monitoring

4 - The HTTP client components TSslHttpCli and TSslHttpRest have new RespMimeType and RespCharset response properties parsed from the Content-Type header to avoid applications needing to parse this headers. Fixed a problem in V9.2 where a missing / was added to the start of the request path, but was not needed for absolute paths used for proxies.

5 - The TIcsIpStrmLog streaming log component has improvements for TCP Server mode when multiple remote clients connect. Previously the same data was sent to all remote clients (the original concept being remote logging), but now applications can send data to specific remote clients, and more easily check which remote client is receiving data. This change means TIcsIpStrmLog can be used as the core of many TCP servers with different protocols, such as the new IcsAppMon sample, see above.

6 - The TSslHttpRest and component has a new way for applications to check SSL certificate chains themselves, ignoring OpenSSL bundle checks, usually for self signed private certificates, maybe checking certificate serials, names or public key. If LogSslVerMethod = logSslVerOwnEvent, a new event OnSslCertVerifyEvent is called so the application can check the chain and change the verify result appropriately.

7 - Improved the ability to customise SSL ciphers if the ICS defaults need to be changed. TSslContext and TIcsHosts have three properties, SslCipherList for TLSv12 ciphers, SslCipherList13 for TLSv13 ciphers, and SslCryptoGroups sets the cipher curve groups allowed (like P-256 or X25519). Beware old SslContexts may include group P-512 which must be corrected to T-521. SSL handshake responses now show the curve group used for OpenSSL 3.2 and later. The OverbyteIcsHttpsTst client sample may be used to test the new cipher options, and they will be read from IcsHosts INI files for servers.

8 - Added a new web server sample OverbyteIcsBasicWebServer1.dpr which is a simplified version of OverbyteIcsSslMultiWebServ ignoring configuration INI files, security features, session data, most demo pages and most logging, and settings for localhost set in code, search for IcsHosts to change IP addresses, etc. This sample should be easier to use as a basis for new web server applications. The existing samples OverbyteIcsSslMultiWebServ and OverbyteIcsDDWebService have a new index.html template page, and default to localhost 127.0.0.1 with an internal localhost SSL certificate, so should always response to https://localhost/ without any INI file changes.

9 - Fixed an HTTP web server problem in V9.2 to avoid repeated redirection for virtual default page /, was adding /// etc.

10 - Updated OpenSSL binary and resource files to releases 3.3.2, 3.2.3 and 3.0.15, only one of which will be linked according to defines.

11 - Restored the sample OverbyteIcsConHttp.dpr which is a console example, now supports SSL by replacing THttpCli with TSslHttpRest, no longer needs any events or a message loop for a single sync request, so a less code than without SSL. Now contacts https://wiki.overbyte.eu/wiki.

12 - A lot of changes have been made preparing ICS for Linux. Corrected loading OpenSSL on Posix, now loads the system supplied OpenSSL 3 DLLs on Ubuntu 22.04. The Linux package now builds correctly, but beware WSocket is not yet supported on Linux so no protocols will work. There is a new IcsPemTest FMX sample that works on Ubuntu 22.04 and which will create ICS signed SSL certificates. Note, MacOS support is disabled pending the new Posix implementation.

The release notes for V9.3 are at https://wiki.overbyte.eu/wiki/index.php/ICS_V9.3

Changes in ICS V9.2 include:

1 - V9.2 is a minor release, fixing a few issues introduced in the last major release, and other bugs located since. There are no breaking or installation changes from V9.1, but if updating from earlier releases please read https://wiki.overbyte.eu/wiki/index.php/Updating_projects_to_V9.1

2 - TIcsMailQueue can now queue a prepared EML file created by another application, or perhaps received by the SMTP Server. Added optional SkipEmpty argument to StartMailQu method so queue is not started unless there are pending emails waiting to be sent. The sample has 'Send Prepared EML File' to queue an existing EML file rather then preparing email with properties.

3 - Improved email MIME decoding by supporting embedded boundaries, usually for multipart/alternative parts, within a multipart/mixed message, using code written 20 years ago but suppressed for some reason. Previously these parts were sometimes left encoded within a part. There is a new property LooseRFC to allow decoding if the boundaries in the body are missing the two required hyphens, usually because the boundary also begins with hyphens. TMimeDecodeEx should now always return the body if no MIME parts are found, and TPartInfo has PLevel which is Part Level, and PInfo which is displayable part information for logs. The MimeDDemo sample has various improvements to test these features.

4 - Fixed a nasty Win64 problem reading EC certificates from the Windows Store, which may have caused server crashes, also reproducible in the PemTool listing the Windows Store. This was due to Win64 bad initialisation of a buffer used for a Crypto API call that failed.

5 - The HTTP client now check the URL always has / at start of the path, ie add it for test.com?query. In the REST client, added a sanity check for RawParameters to encode any spaces, which can break the HTTP request. After a file download completed, check actual file size against response size. The multipart/form-data MimeBoundary no longer includes extra -- at start that are required preceding boundaries within parts, some web servers may have been unable to decode our MIME encoding.

6 - The HTTP server has a new method AnswerRedirect for various redirection responses to a new URL. When accessing the default document in a path without a trailing path delimited /, redirect using 301 to the correct path with delimiter instead of adding it locally and displaying the document which will then incorrectly link to pages in a higher level directory. Using the THttpOption hoAddMissPath redirects if the default document is missing perhaps a template or virtual document. Fixed a bug where authenticated POST/PUT requests always returned a 404 error. Added AnswerBodyTB client response with TBytes binary, similar to AnswerString, tested in the sample by supporting favicon.ico request. Check if the request HTTP version gets corrupted due to spaces in the URL, which are not allowed. The SslMultiWebServ sample has new web pages to test POST/PUT and template authentication.

7 - TSslX509Certs has a new function CertResetDomain to reset a certificate order state to None, if the order process stalls or gets confused due to errors. If AcmeV2StartChallgs fails because there are no pending challenges, reset to order to None so it starts again next time and does not loop.

8 - ICS not longer tries to load OpenSSL RAND_screen function that may be missing from recent DLLs.

9 - TIcsHttpMulti fixes a bug introduced in V8.66 that stopped the application setting authentication, rather than adding it to the URL, and a Win64 free stream bug.

10 - TIcsIpStrmLog correctly counts failed client connection attempts if ping is not used first to check the remote IP address. The sample has a new client Retry Attempts box to test this.

11 - Updated the Snippets sample to use authentication to access some the hardcoded URLs, which started failing after authentication was added to test web server bugs (see above).

12 - Added OverbyteIcsHttpThrd sample to show how to use TSslHttpRest component in a multi-threaded program.

13 - Improved Posix support for Linux and Android, not tested or supported yet. Beware SSL does not correctly load for Posix at the moment.

14 - Added support for a new feature release of OpenSSL 3.3 with {$DEFINE OpenSSL_33} in the Defs.inc file, ICS includes new versions of the active versions, 3.3.1, 3.2.2 and 3.0.14, but no longer includes 3.1 since there are two newer feature versions.

15 - Updated the 'ICS Intermediate Short' SSL certificates, used by ICS to generate temporary server certificates to allow SSL servers to run until a Let's Encrypt or commercial certificate is installed. It now expires after 200 days, 21st December 2024, after which self signed certificates will be used instead, unless a newer 'short' is installed.

16 - Only Delphi 10.41 and 10.42 (10.4 with updates 1 or 2) will install correctly with the new install packages, the original RTM version does not support the package LIB suffix: $(Auto) so you must change it manually for each package to 21.0.

The release notes for V9.2 are at https://wiki.overbyte.eu/wiki/index.php/ICS_V9.2

All ICS active samples are available as prebuilt executables, to allow ease of testing without needing to install ICS and build them all. There are four separate zip files split into clients, servers, tools and miscellaneous samples which can be downloaded from https://wiki.overbyte.eu/wiki/index.php/ICS_Samples

Changes in ICS V9.1 include:

1 - Delphi 10.4, 11, 12 and later now use the same install groups and packages, IcsInstallFmx, IcsInstallVcl and IcsInstallVclFmx, making support a lot easier. Version specific groups remain for Delphi 10.3 and earlier, with new groups D(X)InstallVcl for VCL only replacing the old OverbyteIcs(X) groups, again to simplify support. Dozens of old packages have been removed for this release, so please delete all old groups and packages before installing V9.1, to avoid a mix of old and new packages. Only C++ 10.4, 11, 12 and later are now supported, but untested.

2 - The old samples directory has gone and many of the older and little used samples have been archived to a separate download. The active samples used to test and demonstrate all ICS components are now split into the following paths, in the ICS root directory:

demos-delphi-vcl - 45 VCL samples for Windows.
demos-delphi-extra - four VCL samples that need third party components to build.
demos-delphi-fmx - seven FMX samples for Windows, not yet tested on MacOS.
demos-cpp-vcl - all old C++ samples that have not been tested for 10 years, need help.
demos-data - data files for samples, such as web pages.

All these samples can now be built for Win32 and Win64 platforms. Beware the sample project files (.dproj) supplied are built with modern compilers, and can not be opened by legacy compilers due to new platforms and features, so you MUST delete the .dproj file before opening samples in legacy compilers so the .dproj file will be automatically recreated from the .dpr project file by Delphi.

3 - To ease development, linking and future support, some new units have been added by splitting existing units with multiple components, unfortunately this means many existing projects will need one or more of the new units adding to their uses section. Apologies for the pain, but this should have been done a long time ago. The main change is splitting out much of the SSL/TLS related code from the massive OverbyteIcsWSocket unit to a new unit OverbyteIcsSslBase. Also the OverbyteIcsSslHttpRest.pas has been split with two new units OverbyteIcsDnsHttps.pas and OverbyteIcsSslUtils.pas, to ease linking avoiding circular references. Another new unit OverbyteIcsHtmlUtils.pas now contains functions designed to build HTML pages that were previously split across different units.

4 - Distribution of the ICS OpenSSL files has changed. Earlier ICS versions required the OpenSSL DLLs to be distributed with applications, and a root CA bundle file to verify SSL/TLS connections, and these needed to be loaded using code. There was little standardisation over where the OpenSSL DLLs were located, applications tended to keep their own copies alongside other executables, leading to multiple DLL copies and needing the public variable GSSL_DLL_DIR set to a specific directory before OpenSSL was loaded. Likewise, root CA bundle directories had to be distributed with applications and loaded with code. ICS V9.1 allows five different ways of loading the OpenSSL DLLs:

1. DLLs linked into application as resource files
2.  DLLs loaded from common directory C:\ProgramData\ICS-OpenSSL\
3.  OpenSSL DCU linked into application using commercial YuOpenSSL
4.  DLLs loaded from location specified in public variable GSSL_DLL_DIR
5.  DLLs loaded according to path, may be found anywhere on PC

Which method ICS uses to load OpenSSL depends upon several defines in the .\Source\Include\OverbyteIcsDefs.inc file, please see the readme9.txt file and notes below for details. ICS currently includes resource files for three different OpenSSL releases, 3.0`13. 3.1.5 and 3.2.1, which version is linked is controlled by a define. If the OpenSSL DLLs are linked into the application, they are extracted to a version subdirectory, ie C:\ProgramData\ICS-OpenSSL\3012\ so different applications can use different OpenSSL versions. This happens only once if the files have not already been extracted. When updating existing projects without using any new defines, the ICS old behaviour of methods 3, 4 and 5 above remain with no changes needed.

5 - A common IcsSslRootCAStore component is now created at application start-up, to avoid different components needing their own CA stores to verify SSL/TLS certificates, and for applications to load those stores. The three different CA stores included with ICS are now supplied as resource files, with a define determining which is linked into applications. Another define causes OpenSSL and this store to be loaded at application startup, so OpenSSL is available for all components, without it needing to be loaded again, perhaps repeatedly. Without new defines, a CA Store can be loaded manually into IcsSslRootCAStore. The ICS servers use CA Stores now use IcsSslRootCAStore and no longer load any files specified.

6 - All SSL/TLS servers need a certificate and private key to start, even when testing. Previously ICS supplied some self signed certificates for testing, and also created such certificates automatically if they were missing or if the server was about to order a Let's Encrypt certificate. Accessing such servers for testing using browsers raised various warnings. ICS now has it's own SSL root certificate 'ICS Root CA' and two intermediates, 'ICS Intermediate' and 'ICS Intermediate Short', the last of which includes a private key so can be used to automatically sign new certificates by ICS server applications, rather than just self signed certificates as before. If the 'ICS Root CA' certificate is installed in the Window Store and browser stores, it should stop certificate warnings appearing. ICS applications automatically trust the ICS root, so will give no warnings. The short intermediate has a maximum 100 day expiry, so new versions will be issued regularly. There is a single function CreateSelfSignCertEx that created signed certificates, and another IcsInstallIcsRoot that installs the ICS root into the Windows Store, so easy to use. It is possible to replace the ICS root with your own private root certificate and have servers create their own certificates against that root, for internal networks.

7 - Several client and server components have a new property NoSSL which if set will prevent those components using SSL/TLS for HTTPS or FTPS, even if the application is linked with OpenSSL code. Beware the IcsSslRootCAStore component must not be initialised by the application.

8 - The large OverbyteIcsWSocket unit has been split, by moving TSslContext, TSslBaseComponent, TX509Base and TX509List to a new unit OverbyteIcsSslBase, with only the SslContext callbacks left here since they need access to it, now set in InitSSLConnection instead of InitContext. No longer supporting defines OPENSSL_USE_DELPHI_MM (never used), OPENSSL_NO_ENGINE (deprecated, never used), OPENSSL_USE_RESOURCE_STRINGS (never used), NO_OSSL_VERSION_CHECK (dangerous), DEFINE OPENSSL_NO_TLSEXT (TLS needed everywhere), and LOADSSL_ERROR_FILE (better debugging now). If a connection fails, don't change State to wsConnected briefly before changing it again to wsClosed. Added TSslWsocket SslAlpnProtocols property to specify a list of protocols for clients to send to servers, instead of a similar SslContext property.

9 - OverbyteIcsSslBase is a new unit with TSslContext, TSslBaseComponent, T509Base and TX509List from OverbyteIcsWSocket, also function sslRootCACertsBundle moves here from X509Utils. Added property X509PubKeyTB to TX509Base to get the certificate public in DER binary format as TBytes, from where it may be converted to hex or base64, used for Raw Public Key certificate validation. Made more TX509Base functions and variables public so they can be accessed from other units. Added DHE-RSA-CHACHA20-POLY1305 to TLS/1.2 sslCiphersMozillaSrvTLS12. Added IcsReadTBBio, IcsWriteStrBio, IcsWriteTBBio, IcsSslLoadStackFromP12TB which are internal functions for handling TBytes and certificates, to simplify code (we use too many AnsiStrings for binary data). Added function IcsReportOpenSSLVer to centralise version reporting, optionally adding number of CA root certificates loaded. Saving a private key with a PCKS12 file is now optional. Moved BuildCertFName from WSocketS as IcsIcsBuildCertFName. ICSRootCA.pem and ICS_Intermediate_Short-bundle.pem certificates linked as resources, root is added to IcsSslRootCAStore.

10 - Added new TSslRootCAStore component to OverbyteIcsSslBase derived from TX509List with an Initialise method that loads OpenSSL, then tries to load the internal certificate sslRootCACertsBundle that should be linked into the app, if missing then tries to load DefRootCABundle.pem from C:\ProgramData\ICS-OpenSSL\ or the app path. It also tries to load ExtraRootCABundle.pem which is an optional private root bundle that can be used for private customer or devel roots. Added public IcsSslRootCAStore component created and intialised when this unit is loaded so a common root store is ready for any SslContext or other components. Define OpenSSL_AutoLoad_CA_Bundle can be suppressed to stop OpenSSL and the bundled being automatically loaded, if not needed. SslContext has new property UseSharedCAStore which causes the properties CAFile, CALines and CAPath to be ignored, uses IcsSslRootCAStore instead.

11 - The TX509List class can now load and save PKCS#12 certificate bundle files, smaller than PEM files, added SaveToP12File, SaveToP12TB, LoadAllFromP12File, LoadAllFromP12TB, intended to load a certificate bundle. LoadAllFromPemFile and LoadAllFromPemTB renamed from LoadAllFromFileEx and AddAllFromFStringEx with new versions handling both PEM and PKCS#12 certificate bundle files. TX509List has new method ListCerts that returns one listing line per cert.

12 - The OverbyteIcsLIBEAY and OverbyteIcsSSLEAY units no longer support for OpenSSL 1.1.1 which is end of life. The GSSLEAY_DLL_IgnoreOld/New public variables are currently ignored since only 3.x supported. Added public variable GSSLEAY_RES_SUBDIR which defaults to "ICS-OpenSSL", where OpenSSL files will be saved and accessed if linked as a resource in the application, with a sub-directory for each different version, as mentioned earlier. Added GSSL_CERTS_DIR and GSSL_ROOTS_DIR globals where ICS looks for SSL/TLS certificates and bundles.

13 - Several new defines are added to the .\Source\Include\OverbyteIcsDefs.inc file to determine how OpenSSL is loaded, all those relevant are:

{$DEFINE USE_SSL} - default enabled, link OpenSSL into all components.

{$DEFINE OpenSSL_Resource_Files} - default enabled, link OpenSSL DLLs as resource file into applications, and extract them to shell path CSIDL_COMMON_APPDATA and sub-directory "ICS-OpenSSL" with a version subdirectory, ie C:\ProgramData\ICS-OpenSSL\3012\ . This happens only once if the files have not already been extracted. Ignored for YuOpenSSL.

{$DEFINE OpenSSL_32} - if OpenSSL_Resource_Files is enabled, determines which major and minor version of OpenSSL is linked into the application, 32 is 3.2.x, or 31 or 30. ICS is currently distributed with OpenSSL 3.0, 3.1 and 3.2, the latest patch of each version so 3.0`13. 3.1.5 and 3.2.1, the resources files are in .\Source\, LibV32OpenSSL32.RES for 3.2 Win32, total six resource files, ICS automatically links Win32 or Win64 RES files.

{$DEFINE OpenSSL_ProgramData} - default enabled, but ignored if OpenSSL_Resource_Files or YuOpenSSL enabled. Causes ICS to load OpenSSL DLLs from C:\ProgramData\ICS-OpenSSL\, an alias for C:\Users\All Users\ICS-OpenSSL. ICS is distributed with Win32 and Win64 DLLs for 3.2.1 in .\ICS-OpenSSL which are copied there when building the IcsCommonXXRun package. Note there is no version sub-directory so no version choice. If enabled, overrides the public variable GSSL_DLL_DIR which some applications set to load OpenSSL from a known directory.

{$DEFINE YuOpenSSL} - default disabled. If enabled, compiles the OpenSSL code as a DCU directly into binaries so the OpenSSL are not needed, YuOpenSSL is a commercial product from https://www.yunqa.de/. OpenSSL 3.0 and 3.2 are available for YuOpenSSL.

NOTE: if defines OpenSSL_Resource_Files, OpenSSL_ProgramData and YuOpenSSL are all disabled or missing, ICS loads OpenSSL from the directory specified in the public variable GSSL_DLL_DIR, which is typically set to the application directory. If blank, Windows will search the path for any OpenSSL 3 DLLs, anywhere.

{$DEFINE OpenSSL_CA_Bundle_Small} - default enabled, links a root certificate authority bundle as a resource file into applications, other options are Medium and Large. CA bundles are needed to verify that SSL/TLS certificates are issued by trusted authorities, the resources files are in .\Source\, sslRootCACertsBundle.RES (OpenSSL_CA_Bundle_Small), TrustedCaBundle.RES (OpenSSL_CA_Bundle_Medium) and RootCaCertsBundle.RES (OpenSSL_CA_Bundle_Large).

{$DEFINE OpenSSL_AutoLoad_CA_Bundle} - default enabled. With ICS V9.1 and later, a common IcsSslRootCAStore component is created at application start-up, if this define is enabled OpenSSL will be loaded followed by the root CA bundle RES file according to define OpenSSL_CA_Bundle_Small/Medium/Large. This means OpenSSL is available for all components, without it needing to be loaded again, perhaps repeatedly, and multiple components can share the IcsSslRootCAStore component without needing to load their own CA bundles. If this defined is not enabled, SslRootCAStore.Initialise may be called by the application to load OpenSSL and the CA bundle, which is done automatically by SslContext.InitContext if not done previously.

{$DEFINE AUTO_X509_CERTS} - default enabled. This define enables automatic SSL/TLS ordering from Let's Encrypt in SocketServer and other servers. Unfortunately this adds a lot of other units, HTTPS REST, Json, OAuth2, etc, increasing the size of server applications, so it may be disabled to make server EXE files smaller if certificates are obtained and installed manually.

Except when using the OpenSSL_AutoLoad_CA_Bundle define, OpenSSL still needs to be loaded before any SSL/TLS functionality can be used. This is done automatically by TSslContext and some other components that use OpenSSL, but this means SSL errors like the DLLs not being found may not be raised until a web page is accessed, etc. So it is generally better to load OpenSSL early on in your application, when errors are easier to handle There is a function IcsReportOpenSSLVer that returns the OpenSSL version loaded and where it was loaded from, to help debug loading problems.

When using the OpenSSL_AutoLoad_CA_Bundle define, if the OpenSSL legacy.dll is needed to support old algorithms, which includes most password protected PFX/PCS12 certificates, it must be loaded using LibeayLoadProviders(True, False); since it is not possible to set the GSSLEAY_LOAD_LEGACY global variable early enough.

14 - The TSslWSocketServer class in OverbyteIcsWSocketS has a new property NoSSL that prevents use of SSL/TLS, must be set before server is started. Replaced FX509CAList with public IcsSslRootCAStore. When creating a local SSL/TLS certificate to allow a server to start, ICS now creates a certificate with the IcsHosts.Hosts names signed by an internal ICS intermediate 'ICS Intermediate Short' signed by 'ICS Root CA' which if installed in Windows and browsers will stop certificate warnings appearing. Previously ICS only created self signed certificates. The global GSSL_INTER_FILE may be changed to an alternate intermediate bundle. The ICS bundle has the password 'password' and a maximum 100 day life, so new intermediates will be required regularly, to prevent misuse. Use the function IcsInstallIcsRoot to install the ICS root certificate into the Windows Root Store, needs admin rights for the Local Machine store. Added property ListenAny returns true if any sockets are listening, ie server is running.

15 - The TSslHttpRest component now allows TRestParams to be created as content type 'Form-Data Body' to create MIME multipart/form-data parameters that may include new TParamType of RPTypeFile that specifies a file name whose binary content will be added to the parameters as a file upload, allowing multiple files and extra parameters. File uploading with HttpUploadSimple can now use TRestParams. TRestParams are now built into a TStream rather than a string to allow larger parameter sizes. Added new property MaxLogParams to TSslHttpRest defaulting to 4,096 to restrict the length of params logged before requests with DebugLevel is DebugParams or better, there may be megabytes. Params are now line broken and binary stripped. Added progress information for file uploading, that may take a while, uploads tested to 7GB, beware preparing the form-data content stream may take a few minutes without progress information. Added new property SharedSslCtx which allows an external TSslContext component to be set to the SslContext property (just as with TSslHttpCli) rather than using the internal RestSslCtx automatically. This will be more efficient on memory when using multiple TSslHttpRest components in parallel Added new property NoSSL to TSslHttpRest that prevents use of HTTPS, must be set before any requests. HTTP redirected to HTTPS will fail.

16 - Redesigned TRestParams to build parameters into ParamStream using GetParamStream, to allow parameters including very large files and since the HTTP component needs a post stream rather than a string, mainly for multipart/form-data parameters, see below, GetParams still returns an AnsiString while GetParametersTB returns TBytes. Added new TRestParams content type of PContNone to make them easier to disable, beware ordial values have changed if this saved rather than a literal. Added new TRestParams content type PContFormData to create multipart/form-data parameters, according to RFC7578 which may include multiple binary files and _charset_ part. The TRestParams AddItem method has a new optional ContentType argument, currently used for PContFormData only. Added TParamType of RPTypeFile for binary file content. Added new TRestParams AddItemFile method that takes a full binary file name with optional file size and ContentType, the latter two will be looked up if not supplied, content from file extension and a MIME table. Added new TRestParams FormDataUtf8 property that if true will add a FormData _charset_ part with utf-8 and send all textual content as utf-8 without UrlEncoding. Added GetEstParamSize that returns Int64 estimated size of the parameters, to allow the application to allocate a TFileStream instead of TMemoryStream if massive files are included, typically more than 50MB. Added IcsPercentEncode and IcsPercentDecode to percent encode and decode any non 7-bit characters, ignore charsets. Similar to UrlEncode but does not change spaces or special chars, except %. ExtractURLEncodedParamList has new optional Values parameter than adds all values to the strings as name=value. The ResultSet2Json method of TRestParamsSrv has a new optional query parameter that is added to the Json to assist processing.

17 - The TSslHttpCli component now only call SetSslAlpnProtocols if using Https. If the Location property is cleared during the OnLocationChange event, relocation is stopped, can be used stop relocation from http to Https. When sending proxy CONNNECT request, add ALPN: header (RFC7639) which will be forwarded to target by some proxies, needed for Acme protocol.

18 - The TSslHttpAppSrv application web server has improvements for processing POST data. Added properties PostedDataTB and PostedDataStr to return posted data in easier to use types than an PAnsiChar buffer. Added MaxUploadMB defaults to 200 MBbyte to restrict maximum size of POST or PUT requests. Added MaxStreamMB defaults to 50 MBbyte as the maximum TMemoryStream size before a TFileStream is instead used with a temporary file name. Added PostedDataStream to which POST and PUT content is written which is what TFormDataAnalyser needs, PostedData pointer now points to the stream memory rather than a stack buffer. PUT requests now save uploaded data similarly to POST. These changes allows file uploads larger than memory, up to MaxUploadSize. Added new property NoSSL that prevents use of HTTPS, must be set before server is started. The INI file reads NoSSl, MaxUploadMB and MaxStreamMB. The web server samples have a new postinfo.html page that decodes and displays any parameters passed.

19 - The Web Socket Client class TSslWebSocketCli has a new property WSFullHdrs which when true causes all HTTP request headers to be sent when upgrading a connection to WebSocket, normally only the important headers are sent. Fixed a problem where multiple or partial frames might arrive together, ensure they are corrected assembled. Added new frame state wsfsIncompleteHeader when this happens.

20 - The Web Socket Server class THttpWSSrvConn now skips websocket upgrade if authentication is needed. Fixed a problem where multiple or partial frames might arrive together, ensure they are corrected assembled.

21 - In the OverbyteIcsPemTool sample, when displaying an X509 certificate, show Raw Public Key in base64, should match that of a PEM file with a public key. Allow to save PKCS12 without a private key. Added Basic Constraints 'Root Certificate Authority' tick box that ignores pathlen, 'Certificate Authority' box is now renamed 'Self Signed or Intermediate' and sets pathlen=0 to restrict signing to top level. Removed creating DH Params, not used nowadays with modern ciphers. Added Create Quick Certificates, allows self signed or CA signed certificates to be created with a single button using function CreateSelfSignCertEx. Only uses CommonName, Alt DNS Names, key type and password, and a root CA bundle if the certificate is CA signed, ICS includes a bundle with the file name in GSSL_INTER_FILE. Always creates a PEM bundle with key and intermediate. When installing certificate into the Windows Store, only install key and inter if supplied, and allow all store types, previously always MyStore. Added button 'Install ICS Root in Windows Store' to he Quick box which calls the function IcsInstallIcsRoot to install the ICS root certificate into the Windows Root Store, needs admin rights for the Local Machine store.

22 - The TRestOAuth class has a new OAuthOption of OAopAuthBasic which means use Basic Authentication with client id and secret instead of sending them as parameters.

23 - In the TIcsRestEmail class, Microsoft 365 Rest Email now supports EmailFmtRaw for both GetEmail and SendEmail to receive and send RFC822 SMTP format messages (like GMail) prepared by the TSslHtmlSmtpCli component with HTML content and attachments, and received message can be decoded using TMimeDecodeW, tested using the OverbyteIcsHttpRestTst sample. New TRestEmailType of RestEmailNone where we don't want REST email, beware ordial values changed if saved instead of literals, default now None.

24 - The TIcsFtpMulti, TIcsHttpMulti and TIcsMailQueue components have a new property NoSSL that prevents use of SSL/TLS, must be set before any requests. SslContext now uses the public IcsSslRootCAStore and ignores root bundle.

25 - The TIcsHttpProxy component now supports the CONNECT ALPN: header (RFC7659), to forward ALPN to target. If source sends SSL ALPN, forward it to target. Perhaps optional or at least remove h2 and h3 which we don't support.

26 - With the TSslX509Certs component, made sure certificate extensions are set for server certificate before creating certificate request so international domain name with accents gets processed. Validation now uses public IcsSslRootCAStore and ignores root bundle. The OwnCASign method to sign our own certificates has a new optional OwnCA that creates an intermediate certificate that can sign certificate requests.

27 - In the OverbyteIcsSslX509Utils unit, the function CreateSelfSignCertEx has an extra argument for the file name of a root CA signing bundle, usually an intermediate bundle, that is used to create a CA signed certificate instead of self signed. Password for CA must be same as certificate. Designed for use with public variable GSSL_INTER_FILE which defaults to an ICS signed intermediate allowing servers to issue their own certificates. The SslCertTools class has a new CaCertLines property which returns CA PEM lines, used to create bundle with intermediate. When creating certificates, if BasicPathLen=-1 leave out Basic Constraints pathlen so root certificates can sign intermediates.

28 - The OverbyteIcsHttpRestTst sample has a new 'No SSL/HTTPS' tick box to disable SSL and HTTPS requests, and a new 'Rest Content' type of 'Form-Data Body' to create MIME multipart/form-data parameters that may include new TParamType of RPTypeFile that specifies a file name whose binary content will be added to the parameters as a file upload, in a similar way to the existing 'Upload File' as 'Form-Data' except allowing multiple files and extra parameters. Added 'Form-Data UTF-8 Charset' tick box so form parameters are encoded as UTF-8 rather than HTML characters. TRestParams are now into a TStream rather than an AnsiString to allow larger sizes, tested up to 8GB. Websocket testing will now parse Json if returned, added Send Multi Lines to send two or more lines of text in a single message or as multiple separate messages.

29 - The TMsCertTools class method SaveToStorePfx has a new argument MsCertStore to allow loading into Windows roots store as well as MyStore. Added function IcsInstallIcsRoot to install the ICS Root CA from linked resource into the Windows Trust Store. Also a new method GetOneCert by SHA1 Digest.

30 - In the OverbyteIcsIpUtils unit, IcsLoadMacPrefixes now tries to load a MAC list file from a resource file nmap-mac-prefixes.RES if linked into application, otherwise loads file nmap-mac-prefixes.txt. Likewise the common port list is loaded from a resource file icsportlist.RES if linked, otherwise loads file icsportlist.txt. These changes avoid needing to distribute the files separately with applications, beware they are now automatically linked, in case not needed.

31 - The TIcsTimeClient SNTP component now sends the proper NTP version to the server, we have been sending v6 for 20 years, when v4 is the latest. Added more NTP servers from cloud providers that are more likely to be running than private ntp.org servers. Fixed IcsGetUTCNtpTime always returning midnight due to strange rounding in newer Delphi versions, meant time server sent wrong time.

32 - Historically, ICS has often used AnsiStrings to handle binary data, sometimes custom byte buffers. Modern versions of Delphi now use TBytes (dynamic array of bytes) for binary, so ICS had added many methods and properties using TBytes, mostly with TB added to existing names. There are now TBytes versions of the Jose, hash and digest functions since all input and output is binary: IcsHMACDigestTB, IcsHMACDigestExTB, IcsHMACDigestVerifyTB, IcsHashDigestTB, IcsAsymSignDigestTB, IcsAsymVerifyDigestTB, IcsJoseGetSigTB and IcsJoseCheckSigTB. New utility functions include IcsTBToHex, Base64EncodeTB, Base64DecodeTB, IcsTBytesToString, IcsMoveTBytesToString, IcsTBytesToStringA, IcsStringToTBytes, IcsStringAToTBytes, IcsBase64UrlDecodeTB, IcsBase64UrlDecodeATB, IcsBase64UrlEncodeTB a, IcsBase64UrlEncodeATB, Utf8ToStringTB. Renamed IcsToASCII to IcsPunyToAsci and IcsToUnicode to IcsPunyToUnicode so they don't get used for the wrong purpose. Added IcsFormatHexStr to break long hex string into groups and lines, defaulting to eight chars per group, 64 per line. Added IcsStrRemCntls to replace control codes (< space) in string with ~, optionally leaving line endings, IcsStrRemCntlsA takes an AnsiString or buffer, IcsStrRemCntlsTB is TBytes buffer. Added IcsStrBeakup to break up text into multiple lines of specified length, default 80. Added IcsTimeToZStr to convert DataTime to string hh:mm:ss:zzz. Added IcsResourceGetTB to read TBytes from a named resource. Added IcsResourceSaveFile to save a file from a named resource. Report mobile platforms to IcsBuiltWithEx. Added IcsDataSaveFile and IcsDataLoadFile to save TBytes to a file, and load it from a file, no error reporting.

New Resource Files

As mentioned above, ICS now includes several resource files that are linked into applications, to avoid distributing and loading separate files, these includes OpenSSL DLLs, certificate authority bundles, root certificates and network information lists.

It is intended to issue new ICS releases containing the latest OpenSSL DLLs shortly after new versions are released, which is typically every three months unless serious security fixes require more frequent releases.

The OpenSSL resource files included with ICS come from the OpenSSL zip distribution files at https://wiki.overbyte.eu/wiki/index.php/ICS_Download . ICS currently includes three different versions for two platforms, only one is ever linked into applications according to DEFINES, see earlier. These RES files contain all the DLLs, which are extracted once to version specific sub-directories.

LibV30OpenSSL32.RES
LibV30OpenSSL64.RES
LibV31OpenSSL32.RES
LibV31OpenSSL64.RES
LibV32OpenSSL32.RES
LibV32OpenSSL64.RES

ICS contains three root certificate authority bundle files, the latest versions of which can be downloaded from https://www.magsys.co.uk/download/software/ca-root-bundles.zip, one of which is linked into applications according to defines. The source bundle files are located in .\ICS-OpenSSL/ICS-RootCAs\, the RES files contail the P12 files which are smaller than the PEM versions.

RootCaCertsBundle.RES {$DEFINE OpenSSL_CA_Bundle_Large}
sslRootCACertsBundle.RES {$DEFINE OpenSSL_CA_Bundle_Small}
TrustedCaBundle.RES {$DEFINE OpenSSL_CA_Bundle_Medium}

There are other smaller resource files, all build with BuildICSResFiles.cmd for which the source files are part of ICS,.

ICSCerts.RES (contains ICSRootCA.pem and ICS_Intermediate_Short-bundle.pem)
ICSPortList.RES (contains ICSPortList.txt)
nmap-mac-prefixes.RES (contains nmap-mac-prefixes.txt)

More detailed release notes are at ICS V9.1 Release Notes

More informaion about updating projhects to ICS V9.1

Changes in ICS V9.0 include:

1 - ICS V9 is planned as a long term support release with no new components or major features added, just bug fixes as needed, major changes will be for ICS V10. It uses OpenSSL 3.0 which is supported until September 2026 so applications should have a good life.

2 - To ease introduction to ICS and for existing users looking for new features, the samples have been re-arranged with a new ActiveDemos project group for modern compilers only, that includes about 50 sample projects that between them illustrate and test all the components that make up ICS. All these samples are also available pre-compiled from the wiki pages so they can be easily tested without needing to build them. There is also a new sample OverbyteIcsSnippets that contains small samples of codes for HTTP REST, upload and download, WebSockets, FTP, simple sockets and send email. The unit includes several almost self contained methods each implementing a single functions, which are hopefully easier to follow than the normal samples used to develop ICS and are heavily documented to try and explain usage.

3 - For OAuth2 authentication, TRestOAuth now supports both embedded and standard browsers, the embedded browser gives a better user experience with the window closing automatically once authentication is complete and not needing a local web server. Beware it may not be supported by Windows or end points. Launching a web page into the standard browser may replace a page being viewed, there may be firewall or other problems connecting to the localhost web server and the browser window remains open upon completion. So the end user should ideally be given a choice of which browser to use. There is a new TOAuthBrowser component and TOAuthLoginForm window that uses TEdgeBrowser (Delphi 10.4 and later) or TWebBrowser (no longer supported by Google) to display the login web pages. The LoginHint property is display in the login window and copied to the clipboard so it may be pasted into the login account field. Any applications using OAuth2 and requiring the embedded browser will need updating to add TOAuthBrowser. Only available for Delphi 2007 and later. The embedded browser can be tested with the sample OverbyteIcsHttpRestTst.

4 - Added new WebSocket client and server components. WebSocket is a full duplex TCP protocol for web servers to support interactive web pages, typically dynamic updating such as chat sessions, spell checkers as you type, search hints, etc, using ws:// or wss:// URLs. The client is TSslWebSocketCli which descends from TSslHttpRest with new methods to connect, send text, bytes, a binary stream or pings, and events for new connections, received or sent frames. The WebSocket server uses the ICS web server with a new class THttpWSSrvConn which overrides THttpAppSrvConnection and provides the same WebSocket methods and events as the client, it's a duplex protocol so client and server can send data. The WebSocket client is tested with the sample OverbyteIcsHttpRestTst and the server with the sample OverbyteIcsSslMultiWebServ which has Echo, EchoPing and Chat demonstration URLs.

5 - Added new MQTT protocol client and server components. MQ Telemetry Transport is a lightweight, publish-subscribe, machine to machine network protocol for message queue/message queuing service. The MQTT protocol defines two types of network entities: a message broker and a number of clients. An MQTT broker is a server that receives all messages from the clients and then routes the messages to the appropriate destination clients. An MQTT client is any device (from a micro controller up to a fully-fledged server) that
runs an MQTT library and connects to an MQTT broker over a network. TIcsMQTTServer and TIcsMQTTClient handle the MQTT protocol, tested by sample OverbyteIcsMQTTst which has both client and server,

6 - Made many improvements to the TDnsQuery component, to make it easier to use in other components and applications, adding synchronous methods and support for trying lists of DNS servers where one or more don't respond. It is also easier to access the arrays of different DNS answers. Added AAAALookup method for IPv6, similar to ALookup, sets array of IPv6 addresses. Allow lookup using multiple DNS servers if one or more fail, from a supplied list or internal public server list including Google, Cloudfare, OpenDNS and others, only works with the new sync methods. Added OnLogEvent primarily for debugging multiple requests and servers, may mostly be ignored.

7 - Added new components TIcsDomainNameCache and TIcsDomNameCacheHttps designed to simplify forward domain name and reverse IP address lookup in applications, to avoid needing use of TWSocket or TDnsQuery components often several to support parallel lookups. There are synchronous methods that wait until a response is received and asynchronous methods that return immediately with an event called when the response is available. Lookups may be performed using uses winsock so results come from the operating system cache, using the TDnsQuery component to make requests to specific servers or public DNS servers, or using DNS over HTTPs for secure lookups. Up to 100 parallel lookups are supported although defaults to five, with waiting lookups queued and performed first in, first out. TIcsDomainNameCache is mainly for use with diagnostic components but also for servers logging remote access. Used in the OverbyteIcsNetTools, OverbyteIcsNetMon, OverbyteIcsBatchDnsLookup, OverbyteIcsDDWebService and OverbyteIcsSslMultiWebServ sample applications. Could potentially be used in other high level applications to avoid using the operating system DNS cache.

8 - ICS servers and clients can now use SSL/TLS certificates from the Windows Store instead of PEM and PFX disk files, with some limitations. PEM and PFX ICS has long being able to read certificates from the Windows Store, but reading some private keys failed due to Windows APIs issues, for which a workaround has now been implemented so LoadFromStore is now able to read EC keys as well as RSA. TMsX509List allows all certificates and private keys in a store to be loaded, with the new FindBest method finding a certificate matching a search string, checking common name, part friendly name or any alternate domain name, selecting the one with the latest expiry if more than one. If the certificate name is a wildcard (*), this will be matched with any first node For clients, this is illustrated in the sample OverbyteIcsHttpRestTst to select a client certificate, for servers, IcsHosts has a new property SslLoadSource which can be set to CertWinStoreUser or CertWinStoreMachine. with former causing the store to be searched for the host name. Note the application will need administrator rights to load certificates from the Local Machine store, it will also not read keys from external hardware devices, and probably not in Trusted Platform Modules either. The sample OverbyteIcsPemTool can be used to view, select and extract certificates from the Windows Store.

9 - There are a lot of general stability improvements in various low level components, protecting inherited class destroy methods from exceptions at higher levels to avoid memory leaks, such as closing a socket left open before destroy, particularly with Win64 applications. Generally free objects instead of destroying them. IcsX509VerifyErrorToStr now checks OpenSSL is loaded to avoid an exception when it's called. If neither OpenSSL DLL can be found, report both names instead of just oldest. Ensure OpenSSL is loaded before using hash functions.

10 - For ICS servers, increased the default IcsHosts security level to sslSrvSecHigh for TLS/1.2 minimum, with certificate key size 2,049 bits and SHA-256 digest. Added CliCertMethod to IcsHosts to allow specific hosts to request a client SSL/TLS certificate rather than all hosts.

11 - HTTP clients have a number of improvements. Some servers allow the GET and DELETE requests to have content similarly to PUT so allow this if new Options httpoGetContent is specified. Beware to set SendStream to nil if no content is intended. Try to prevent the header and content from being sent as separate TCP packets which may confuse some middleware. Always check if SendStream exists before accessing it. Added OnSyncRequestWait event called while waiting for sync operations to finish, so it can be stopped before timeout expires. Improved EHttpException messages with more detail. In TSslHttpRest, when a client certificate is requested, check it has a private key and log some information about it, better logging if no certificate. The sample OverbyteIcsHttpRestTst has a new Client SSL Certificate drop down box with options to load the certificate from a PEM/PFX file, or from the Windows Current User Store or Windows Local Machine Store (admin rights needed). A real application could offer a selection of which certificate to use as browsers do. Added ics-client-test.pem as default Client SSL Certificate, for testing against ICS servers.

12 - FTP clients have a number of improvements. Added NoopAsync method sends NOOP no operation command to try and keep the control connection alive during long transfers, note there is no sync version and the response is ignored (but logged). Beware some older FTP servers may treat NOOP as illegal and fail the transfer (including ICS FTP server V8.70 and earlier). NOOP is sent by TIcsHttpMulti defaulting to 10 minutes. Without this change, FTP transfers to some public servers are failing after two or more hours due to the control connection having been closed by a router or firewall somewhere for inactivity, now tested OK with 50GB uploads (VM images). Added UTF8_ON and UTF8_ONAsync commands as an alternate to using Opts command with an argument. Increased sync timeout to 30 seconds and make sure reset with Progress more often so requests don't timeout. Fixed unicode compiler bug parsing Cmlsd/XCmlsd command response. Don't compress iso, pdf, vhd, vhdx files by default. Disable MD5/CRC32 by default, connections are now reliable and they are slow.

13 - The FTP server has a fix to allow the NOOP keep-alive command to be sent while a file transfer command is being processed without causing it to fail. Added exception handling generating directory listings due to strange directory inputs by hackers causing a problem. Give msgNoPortPsv response if PORT, EPRT, EPSV or PASSIVE command has not been sent when a directory command requiring a data channel is received.

14 - When ordering SSL/TLS certificates from Let's Encrypt, the local web server can now listen on both IPv4 and IPv6 addresses for domains with both. Removed OAuth2 authentication code from the component and get the tokens using and event. Simplified checking DNS challenges with the new TDnsQuery sync methods. Added MsCertLoc property to specify which Windows Certificate Store to save certificates if OutFmtWinStore is specified.

15 - When sending SMTP HTML emails, the EMailImages property may be used to add streams as well as files, instead of ImageStream and StreamArray which never worked properly. Use EMailImages.AddObject(filename, TStream) where file name will be used instead of opening the file (the file need not exist).

16 - The proxy server component has a new OnSrvSslHandshakeDone event called when new client connects to proxy server allowing application to check for a client certificate and abort the connection, needs CliCertMethod to be set in IcsHosts to sslCliCertRequire or sslCliCertOption, so a client certificate is requested. Fixed auto certificate ordering reading well-known file on unicode compilers.

17 - OverbyteIcsWSocket includes various new utility functions. WSocketSockAddrToStr converts TSockAddrIn6 with IPv4 or PIv6 address to a string, WSocketIPAddrToSocAddr convert a string IPv4 or IPv6 address into TSockAddrIn6, WSocketFamilyToAF to find family for Windows APIs from TSocketFamily, WSocketIPv6Same to compare two TIcsIPv6Address. Added an overloaded ReceiveTB that returns a TByte instead of the received size, it also no longer fails if the buffer is not initialised.

18 - OverbyteIcsUtils corrects RFC3339_DateToStr to add colon to time zone, RFC3359 requires +00:00, ISO also accepts +0000. Added StringToUtf8TB convert string to TBytes, IcsTextOnStart case insensitive text at start of line, and IcsTBytesToString to convert TBytes to unicode string. IcsWcToMb and IcsMbToWc now use cross platform RTL functions instead of OverbyteIcsIconv and USE_ICONV which have been removed. IcsIconvNameFromCodePage is now POSIX instead of USE_ICONV.

19 - Added ICS Internet Packet Monitoring Components which display internet packets using raw sockets or Npcap NDIS driver, similarly to Wireshark. They are based on Magenta Systems Internet Packet Monitoring Components but updated with IPv6 and new filtering by protocol and IP address to restrict the amount of data being captured. TIcsMonSocket in OverbyteIcsMonSock provides internet packet monitoring using raw sockets. TIcsMonPcap in OverbyteIcsMonPcap provides internet monitoring using the Npcap NDIS driver. The class TIcsMonFilterClass filters captured packets. There is a new sample OverbyteIcsNetMon that is a simplified Wireshark, displaying captured packets or just totaling traffic. Packets may be captured to a textual log for easy saving or to a grid for improved display including examining each packet separately. Filters include all local IPs including broadcast and multicast, or specific protocols or services, in all case either accepting or rejecting packets according to the filter settings, dynamically during capture. This makes it easy to ignore a lot of local LAN traffic from appliances that can obscure higher level traffic.

20 - Added Internet Protocol Helper Component for Windows, updated from the Magenta Systems units with full IPv6 support and new components. TIcsIpChanges in OverbyteIcsIpHlpApi monitors IP address changes and calls an event for new IPs configured or old ones removed, useful for servers where the listening address suddenly disappears. TIcsNeighbDevices in OverbyteIcsIpHlpApi builds a historic LAN MAC device and IPv4 and IPv6 address table using ARP,neighbourhood and IP range scanning with reverse host lookup. Both are tested with sample OverbyteIcsNetTools. There are many new IpHlp functions including IpHlpAdaptersInfo, IpHlpNetworkParams, IpHlpConnsTable, IpHlpTCPStatistics, IpHlpIfTable2, IpHlpAdaptersAddr, IpHlpIpAddrTable, IpHlpIpNeighbTable, IpHlpIPForwardTable, IpHlpGetRouteProtocol and IpHlpIpPathTable, all of which return various Windows networking tables and information that is useful for diagnosing network problems, all illustrated by the sample OverbyteIcsNetTools. For completeness, the sample also includes other ICS components, for Whois, NsLook, Ping and Trace Route. Other new functions include IpHlpGetDnsServers to get a list of DNS servers for this PC, IcsGetMacVendor which uses the Organizationally Unique Network Interface Identifier nmap-mac-prefixes.txt file to get the MAC adaptor vendor, very useful for identifying strange IoT devices on a LAN.

21 - Updated the OpenSSL DLLs included with ICS to 1.1.1v and 3.1.2. Updated all the certificate root stores. Also supports 3.0.10 which can be downloaded from the wiki pages.

22 - Added two new FMX samples, IcsHttpRestTstFmx and IcsSslMultiWebServ, converted from the VCL versions, rather messy since no proper TRadioGroup in FMX so changed to TlistBox without a caption, TListView exists in FMX but without columns so they become TStringGrid, TGrid would be better except it uses virtual data (which the help fails to mention) which needs a lot of rewriting. A few more FMX samples using new ICS components will be added for the next release

Detailed V9.0 Release Notes
Detailed V8.70 Release Notes
Detailed V8.69 Release Notes
Detailed V8.68 Release Notes
Detailed V8.67 Release Notes
Detailed V8.66 Release Notes
Detailed V8.65 Release Notes
Detailed V8.64 Release Notes
Detailed V8.63 Release Notes
Detailed V8.62 Release Notes
Detailed V8.61 Release Notes
Detailed V8.60 Release Notes
Detailed V8.58 Release Notes
Detailed V8.50 Release Notes

Now part of ICS V8.60 and later, as TIcsMailQueue

Magenta Systems Mail Queue Component
Version 2.5 - 26 Nov 2018 (2,414,770 bytes)

Magenta Systems Mail Queue Component has two main benefits over a simple TSslSmtpCli component: it supports extended retries over many hours or days, and supports multiple SMTP relay servers or looks-up MX servers, while alleviating the need for the application to handle retries. The component also allows HTML mail to be sent using SSL, something THtmlSmtpCli does not currently support. Mail is queued to disk, so retries will continue if the application is restarted.

TMagMailQueue is designed to prepare, queue and send email. Preparing the email is done using the ICS THtmlSmtpCli component so it may be plain text or HTML email with one or more file attachments.

Once the mail properties in QuHtmlSmtp have been specified, it is queued using the QueueMail method which saves it to an EML spool file.

The component runs a thread which checks the queue for new EML spool files, and attempts to forward them to one or more SMTP Mail Servers using TSslSmtpCli, optionally with SSL. If mail delivery succeeds, the spool file may be deleted or moved to an archive folder. If mail delivery fails, the spool file remains in the queue and further attempts are made separated by the times in minutes listed in the RetryList list. If all delivery attempts fail, the spool file may be deleted or moved to a badmail folder.

Note that some email servers support grey listing and reject the first email attempt from a new sender but allow a retry 10 or 15 minutes later, something that is very effective in blocking spam emails (since they don't usually retry).

If multiple mail servers are specified, delivery is attempted once using each server, for each retry attempt. Each mail server is specified as TMailServer and there is no limit to the total.

Each time the queue is updated or a delivery attempt made, the queue is saved to file in the control folder, so the component may be stopped and restarted with failed attempts continuing.

The EML spool files are compatible with those created by many Microsoft email applications such as CDO, and the AddtoQueue method can also be used to queue existing EML files with the queue details specified in MailQuItem.

Note, this component is intended for sending low volume email from individual Delphi applications, with more flexibility than a simple TSslSmtpCli component. For use as a heavy duty SMTP server, queue processing could be improved to avoid moving records around as much or saving them to disk as often, and mail bodies could be read as required from disk instead of being read entirely to memory first.  A mail pickup folder could be added which is scanned for new EML files.

Files and Folders Used

The TMagMailQueue component heavily uses disk files, in different sub-directories within the mail root directory specified in property MailQuDir, these are:

• control - contains MailQuItems.Ctl a single row file with the next message item number, and MailQuItems.Hdr which is a CSV file containing one row for each mail item still in the queue.
• spool - contains any queued email files, named in the format item00000001.eml with the number increasing, taken from MailQuItems.Ctl
• archive - if ArchiveSent property is true, once an email has been successfully sent it is moved into the archive directory
• badmail - if DeleteFailed property is false, once an email has exceeded all the retry attempts it is moved into the badmail directory, from where it may be manually requeued if necessary

If logging of sent email is specified, the default file name FileQuSent property is MailQuSent-yyyymmdd.log inb CSV format similar to MailQuItems.Hdr.

A demo application mailqudemo.exe illustrates simple email queuing.  The zip contains the EXE demo and required SSL files.

Release Notes

18th January 2011 - 1.0 - first public release. Not yet tested with Delphi 2009 or later.

2nd March 2011 - 1.2 - automatically create mailqueue directory in demo application, removed missing uses statement. Support queuing mail with OwnHeaders bypassing htmlmail. Log event definition changed.

11th August 2011 - 1.2 - updates subroutines for Win64 support, removed one unneeded unit from uses.

5th Oct 2011 - 1.3 - Debug logging works properly Don't retry emails that fail too large for server (error 552)

11th Sept 2012 - 1.4 - ICS V8, IPv6

23rd March 2013 - 1.5 - Added Mail Server SocketFamily and LocalAddr6 for IPv6

10th Dec 2014 - 1.6 - Better SSL handshake reporting

27th Oct 2015 - 2.0 - requires ICS V8.19 October 2015 or later.
Check and report SSL certificates using PEM file or Windows Cert Store
Allow three SMTP servers to be specified for each email in queue
Lookup DNS MX records and send to those SMTP servers
Queue keeps last response or error in queue
Mail completed log (same CSV format as queue)
Queue changed event to tell client something is happening
QueueMail method now returns item number (not boolean)
New UnQueueMail method to remove item number from queue
Demo save settings in INI file
Demo new View Mail Queue window to see what's waiting
Added SMTP Send Method, relay, specific or lookup MX mail servers
Added HELO Sending Host Name may be needed if using MX mail servers

Warning - if using MX DNS servers and multiple recipients, need to queue mail multiple times !!!! This will be fixed real soon.

7th July 2016 - 2.1 - requires ICS V8.30 July 2016 or later.
Support SSL enhancements in ICS for OpenSSL 1.1.0
Don't change SSL directory, let application control it
Use default SSL root bundle if none specified

1st December 2016 - 2.2 - requires ICS V8.39 November 2016 or later.
Better error handling.
Use OpenSSL host checking.
Fixed bug that meant failed email was not deleted from queue.
Don't queue email without recipients.
Use timer to update windows to avoid problems with mass email performance.

6th March 2017 - 2.3 - requires ICS V8.43 March 2017 or later.
Simplified SSL certificate reporting.

11 Mar 2017 - 2-4 - Added WaitSend to wait until everything sent.

22th Jun 2018 - 2.5 - requires ICS V8.55 20 June 2018 or later.
Added RetryWithoutSsl which retries an SSL failure without SSL.
Added SslCliSecurity to set client security level.
Using IcsWndControl for threaded message handling.
SendSmtpClient now created new for each attempt in case of prior faillure causing terminal corruption.
If SSL certificate verify fails, next attempt is another server.
Supports TLSv1.3 with OpenSSL 1.1.1.

26th November 2018 - 2.5 - tested with ICS 8.58
Added final OpenSSL 1.1.1a DLLs, recompiled.

Now part of ICS V8.60 and later. 

Now part of ICS V8.60 and later, as TIcsIpStrmLog

Magenta Systems IP Log Streaming Component
Version 2.8 - 14 Dec 2018 (4,156,649 bytes)

TMagIpLog is designed for IP stream logging, using TCP Client, TCP Server, UDP Client or UDP Server protocols, sending simple text lines across a network so they may be displayed or written to disk remotely. The component allows two way communication with TCP and UDP, so may also be used for simple protocols such as communication between two applications. The component supports multiple client sockets so may be used to send data to two or more different remote servers at the same time.

 For TCP and UDP clients, the component will optionally ping the remote computer first before opening an IP connection to allow faster failure retries and some confirmation that UDP may work. TCP client provides repeated connection retry attempts, including re-establishing a lost connection. UDP client will optionally keep pinging the remote during a connection to ensure it's still there. UDP server sends data to the IP address and port from which it last received data. TCP server supports multiple remote clients connecting. Received data is parsed for various line endings optionally removing control characters and triggering an event for a received line. The only other two events are optional, one for state changed when starting and stopping, the second offering progress information and errors. 

The component supports both IPv4 and IPv6, host name lookup for TCP and UDP Client, and SSL connections for TCP Client and TCP Server, including remote server certificate checking using either a local PEM bundle root file or the Windows Certificate Store.

A demo application testiplog.exe illustrates use of TMagIpLog as a TCP or UDP client or server, and both in the same program sending data locally. The same component may be used in a client or server application, to send or receive.

The Magenta Systems ComCap application may also be used to capture IP streams to files or a database.

Using TMagIpLog:

1 - Drop the component onto a form (or create it in code, see testiplog.exe).

2 - Specify LogProtocol as one of logprotUdpClient, logprotUdpServer, logprotTcpServer, logprotTcpClient.

3 - For client protocols, specify RemoteHost (name or IP address) and RemoteIpPort, CheckPing true if ping to be used, RetryAttempts to non-zero if continual retries not needed, RetryWaitSecs for delay between retries .

4 - For server protocols, LocalIpAddress is 0.0.0.0 to listen on all local addresses, LocalIpPort must be non-zero.

5 - For sending data, AddCRLF to false if line already have terminating characters, UdpNoCRLF to false if UDP should send CRLF.

6 - For receiving data, LineEndType to one of lineendCR, lineendLF, lineendCustom (set in hex in CustomLineEnd) or lineendPacket (for UDP), then MaxLineLen if a line should be returned before lineend is found, normally non-ASCII characters are removed, set StripControls to false if they should be replaced by spaces, RawData to true if CR, LF, FF and control characters should not be removed.

7 - Assign onLogRecvEvent if data is to be received, onLogChangeEvent if tracking of start and stop is needed, onLogProgEvent if progress information is needed for logging.

8 - Call StartLogging. The LogChangeEvent and LogProgEvent will trigger when LogState changes to logstateOK when data may be sent.

9 - To send a line, if function GetAnyStateOK is true, call SendLogLine. MaxSendBuffer specifies the amount of data that can be buffered otherwise SendLogLine will fail.

10 - Received data will trigger LogRecvEvent once per line.

11 - Call StopLogging to stop. Buffered data may continue to be sent after close, keep calling CheckStopped until true when it's really finished and component may be destroyed.

12 - To send an unlimited size stream, create a stream in the application with TBufferedFileStream or TFileStream, and pass it to SendStream.  LogState changes to logstateOKStream while it's being sent, then back to logstateOK as it finishes, the application should then free the stream.

13 - There is no specific handling for receiving a stream, textual data will be handled according to the normal line end properties, and can be saved to another stream in LogRecvEvent. Binary data is more problematic, set RawData to true and MaxLineLen to get a buffer load at a time, but the last buffer load will need to be extracted with GetPartialLine using a timeout, this is called automatically when the connection is closed.

14 - To send to multiple clients, set MaxSockets to the number needed, then use the function SetRemotes to specify the remote host and port for each socket number, base 0. The events all return Socnr to indicate which socket. MaxSockets also specifies how many remote clients can connect to TCP Server, but note that Socnr is dynamic and changes as remote clients come and go.

15 - To support SSL on TCP/IP client or server, drop an TSslContext component on the form, assign it to the LogSslContext property and set the ForceSsl property to true. For better performance, set LogSslSessCache to a TSslAvlSessionCache component.

16 - For SSL TCP Server, the SslContext component must have the SslCertFile and SslPrivKeyFile properties set to the file names of an SSL certificate and Private Key PEM files respectively, and SslCipherList set to sCipherMozillaSrvBack for strong but backward compatiblle cipher support. The component includes sample self signed certificate and password files iplog-cert.pem and iplog-prvkey.pem, and you can create your own with the ICS SSL sample application Pemtool, or buy commercial PEM certificates.

17 - For SSL TCP Client, the SslContext component must have the SslCAFile property set to the file name of a PEM root certification authority file containing trusted root certificates. Such a file is supplied with the component RootCaCertsBundle.pem containing various root certificates covering most major registries. SslContext SslCipherList can be left as the default to allow connection to any server. The LogSslVerMethod property can be logSslVerNone to skip certificate verification, logSslVerBundle to check using the CA bundle file or logSslVerWinStore to check using the Windows certificate store (a little slower, bur maybe more certificates). To check if certificates have been revoked set LogSslRevocation to true, beware this needs public internet access and can be very slow or fail. LogSslReportChain set to true reports certificate details checked.

Release Notes

18th August 2007 - 1.1 - using OverbyteIcsFtpSrvT instead of OverbyteIcsLibrary, UDP receive packets may be from multiple hosts, always keep IP.

5th August 2008 - 1.2 - made compatible with ICS V7 and Delphi 2009. Note only supports ANSI with Delphi 2009.

20th August 2009 - 1.3 - fixed problem with MaxSockets being reported as closed in the event when only one was open, tested with Delphi 2010.

9th August 2010 - 1.4 - removed cast warnings with Delphi 2009 and later

22nd Sept 2011 - 1.5 - added SndBufSize and RcvBufSize to increase buffer sizes and speed

11th Sept 2012 - 1.6 - better error for too many clients with server added CurSockets property for current number of server sockets

7th July 2014 - 2.0 - now only ICS 8 and later, using new ICS ping. 
Added IPv6 and SSL support, including server certificate checking.
Added host name support for UDP and TCP client with DNS lookup.
Added LogProtocols suffixed 6 for IPv6.
Cleaned up some progress messages, identify error progress events.
Removed line length limit of 1024 that was not checked.
Added send a stream of unlimited length.
Get buffered partial received line during close.
Default line end is LF instead of CR so UNIX files are processed.

13th July 2015 - 2.2 - requires ICS V8.18 June 2015 or later.
Added better SSL handshake error reporting.
Added lineendCRLF, only support FF as lineend if using CR.
Added Debug Info button for ICS info level logging.
Added SSL Server DH Params, set ECDHCurves, both for ECDH ciphers.
Note OpenSSL no longer support dhparam512, minimum is 768 bits.

23rd Oct 2015 - 2.3 - requires ICS V8.19 October 2015 or later.
Better SSL client and server certificate reporting.

8th July 2016 - 2.4 - requires ICS V8.30 July 2016 or later.
Fixed certificate reporting typo.
Removed TBufferedFileStream, not needed.
Added SrvTimeoutSecs to close idle server sessions, note needs ICS V8.30  or later to fix a SSL bug that stopped SrvTimeoutSecs working.
Added Socket property to get current socket, mainly for statistics
Report session length and data xmit/recv before closing

23rd Nov 2016 - 2.5 - requires ICS V8.39 November 2016 or later.
Added GetSendWaiting to check how many bytes of send data not yet sent.
Increased default MaxSendBuffer size to 64K.
Added property TotRecvData total data received since connection, or when method ResetRecvData was called.
Added property MaxRecvData which causes onLogRecvEvent to be called when that length has been received. May be used for fixed length binary packets or where received data contains a content length such as a HTTP response header followed by binary data.
Server takes exclusive access of addr/port.
Fixed bug with multiple clients not using correct port.
Added SSL Server Name Indication support.
Check multiple client SSL host names correctly.
Removed USE_SSL so SSL is always supported.
Removed TX509Ex now using TX509Base.
Using OpenSSL certificate verification host checking.
Server now supports LogSslReportChain to report server certificates, checks expired and reports chain.

7th March 2017 - 2.6 - requires ICS V8.43 March 2017 or later.
set IcsLogger for context so it logs more stuff.
Simplified reporting SSL certs in client handshake.
Improved validation of server certificates.
Use threaded DNS lookup.

22nd June 2018 - 2.7 - requires ICS V8.55 20 June 2018 or later.
Support TLSv1.3, no real changes.
Don't start SSL handshake twice.
Cleaned up SSL error handling.
Added SslCliSecurity to set client security.

14th December 2018 - 2.8 - tested with ICS 8.58
Added final OpenSSL 1.1.1a DLLs, recompiled.
Removed madexcept.

Pending major changes to use IcsHosts in 3.0.

Now part of ICS V8.60 and later.  Uses IcsHosts.

Now part of ICS V8.60 and later, as TIcsWhoisCli with a new sample application

Whois Component and Demo
Version 1.0 - 2 Nov 2005 (254,079 bytes)

A Whois component and demonstration application.  Whois is a protocol to interpret a remote server for information about a domain name or an IP address, and return textual information about 'owner' of the name or address.  The demo application interprets the result and will perform a secondary query to another Whois server if necessary.

Now part of ICS V8.60, as TIcsTimeClient and TIcsTimeServer with a new sample application

SNTP Time Server and Client Components
Version 1.0 - 9 Mar 2006 (11,405 bytes)

TTimeServ is an updated version of Nathan Anderson's time server component adding SNTP support.  TWSTimeClient is an updated version of Chris Barber's time client component adding SNTP support and functions to change the PC UTC time.  SNTP provides time correction with fractional seconds, unlike the earlier Time protocol that is round seconds only.

Magenta Systems Ltd, 9 Vincent Road, Croydon CR0 6ED, United Kingdom
Phone 020 8656 3636, International Phone +44 20 8656 3636
https://www.magsys.co.uk/
Copyright © 2024 Magenta Systems Ltd, England. All Rights Reserved.