(Free password required)
27th November 2023 - OpenSSL 3.2.0 Windows binaries released.
9th November 2023 - OpenSSL 3.1.4 and 3.0.12 Windows binaries released.
th November 2023 - ICS V9.0 already has Delphi 12 packages.
21st September 2023 - OpenSSL 3.1.3, 3.0.11 and 1.1.1w Windows binaries released. Note the binaries are now digitally signed by 'Magenta Systems Ltd' .
22nd August 2023 - ICS V9.0 released, many new components and features
August 2023 - Magenta Systems Internet Protocol Helper Component and Magenta Systems Internet Packet Monitoring Components are now part of ICS v9, with major upgrades and new samples, with full IPv6 support.
6th July 2023 - Updated PEM Bundle CA Trusted Store Files
There is a new ICS support forum at https://en.delphipraxis.net/forum/37-ics-internet-component-suite/ to replace the old TWSocket mailing that stopped working a few years ago.
This page contains various components written by Magenta Systems Ltd to extend the François Piette's Internet Component Suite (ICS) version 8 from http://www.overbyte.eu/. ICS v8 supports Delphi 7, 2006 to 2010, XE to XE8, 10 Seattle, 10.1 Berlin, 10.2 Tokyo, 10.3 Rio,10.4 Sydney, 11 and 12. Note that all ICS versions now include SSL free of charge.
All ICS files should normally be accessed from the ICS Download Wiki page but the important files are available here as well.
The latest versions of ICS may be downloaded from the ICS SubVersion server using a subversion client such as TortoiseSVN. Once your SVN client is installed, you can browse to svn://svn.overbyte.be/ics, svn://svn.magsys.co.uk/ics or https://svn.overbyte.be/svn/ics or https://svn.magsys.co.uk/svn/ics. The SVN login user code is ics and password is ics for read access.
Nightly ICS v9 for Delphi 7-2010, XE-XE8,
10 Seattle, 10,1 Berlin, 10.2 Tokyo, 10.3 Rio, 10.4 Sydney, 11 and 12
Latest ICS V9.0 Release
OpenSSL Binaries 3.2 Win32
OpenSSL Binaries 3.1 Win32
OpenSSL Binaries 3.0 Win32
OpenSSL Binaries 1.1.1 Win32
OpenSSL Binaries 1.0.2 Win32
Sources of CA Trusted Stores
SSL/TLS trusted root certificate bundles and always changing, annually perhaps for major changes, although Microsoft officially Windows roots every two months. The Common CA Database (CCADB) https://www.ccadb.org/ is a repository of information about Certificate Authorities (CAs), and is used by a number of different root store operators to manage their root stores.
But it's not easy to create root bundles from CCADB and another developer got frustrated with updating roots, and created a Trust Stores Observatory Git repository: https://github.com/nabla-c0d3/trust_stores_observatory which contains over 649 root certificates and lists of which trust store contain which roots by different operating systems. But even this does not contain certificates in a form easily used by OpenSSL, so Magenta Systems Ltd has written a small tool that converts the YAML files from TSO into PEM bundle files, one each for the different operating systems.
CA PEM Bundle - 6 Jul 2023 (1,475,840 bytes)
There are six different PEM CA bundle files, built from the Trust Stores Observatory Git repository in July 2023:
apple.pem - 165 Certificates
Each certificate is prefixed by it's description, issuer fields, expiry, public key type and SHA256 hash, so the bundles are self documenting rather than being just cryptic base64 blocks. These PEM bundles may be loaded into an OpenSSL context as a root store. Magenta Systems Ltd will periodically update these bundles, as needed. The files are all UTF-8 with a BOM. While the certificates are base64 encoded, the added comments may include Unicode characters for non-English issuers.
The zip file contains two versions of each bundle, the name above and one ending with -clean.pem which omits all the added textual comments so is smaller and less likely to cause problems with non-English characters. There are also -titles.txt and -fprints.txt files which are one line per certificate listing the main details, and fingerprint in the latter file. There are also changes files for the Microsoft Windows bundle that indicates which certificates were removed or added with each update.
Note the ICS distribution download (see above) contains three CA Trusted Stores, two as PEM bundle files, one in a source unit, and access to the Window Certificate Store directly, see FAQ_SSL/TLS_Certificate_Authority_Root_Stores for more information.
Changes in ICS V9.0 include:
1 - ICS V9 is planned as a long term support release with no new components or major features added, just bug fixes as needed, major changes will be for ICS V10. It uses OpenSSL 3.0 which is supported until September 2026 so applications should have a good life.
2 - To ease introduction to ICS and for existing users looking for new features, the samples have been re-arranged with a new ActiveDemos project group for modern compilers only, that includes about 50 sample projects that between them illustrate and test all the components that make up ICS. All these samples are also available pre-compiled from the wiki pages so they can be easily tested without needing to build them. There is also a new sample OverbyteIcsSnippets that contains small samples of codes for HTTP REST, upload and download, WebSockets, FTP, simple sockets and send email. The unit includes several almost self contained methods each implementing a single functions, which are hopefully easier to follow than the normal samples used to develop ICS and are heavily documented to try and explain usage.
3 - For OAuth2 authentication, TRestOAuth now supports both embedded and standard browsers, the embedded browser gives a better user experience with the window closing automatically once authentication is complete and not needing a local web server. Beware it may not be supported by Windows or end points. Launching a web page into the standard browser may replace a page being viewed, there may be firewall or other problems connecting to the localhost web server and the browser window remains open upon completion. So the end user should ideally be given a choice of which browser to use. There is a new TOAuthBrowser component and TOAuthLoginForm window that uses TEdgeBrowser (Delphi 10.4 and later) or TWebBrowser (no longer supported by Google) to display the login web pages. The LoginHint property is display in the login window and copied to the clipboard so it may be pasted into the login account field. Any applications using OAuth2 and requiring the embedded browser will need updating to add TOAuthBrowser. Only available for Delphi 2007 and later. The embedded browser can be tested with the sample OverbyteIcsHttpRestTst.
4 - Added new WebSocket client and server components. WebSocket is a full duplex TCP protocol for web servers to support interactive web pages, typically dynamic updating such as chat sessions, spell checkers as you type, search hints, etc, using ws:// or wss:// URLs. The client is TSslWebSocketCli which descends from TSslHttpRest with new methods to connect, send text, bytes, a binary stream or pings, and events for new connections, received or sent frames. The WebSocket server uses the ICS web server with a new class THttpWSSrvConn which overrides THttpAppSrvConnection and provides the same WebSocket methods and events as the client, it's a duplex protocol so client and server can send data. The WebSocket client is tested with the sample OverbyteIcsHttpRestTst and the server with the sample OverbyteIcsSslMultiWebServ which has Echo, EchoPing and Chat demonstration URLs.
5 - Added new MQTT protocol client and server components. MQ Telemetry
Transport is a lightweight, publish-subscribe, machine to machine network
protocol for message queue/message queuing service. The MQTT protocol
defines two types of network entities: a message broker and a number of
clients. An MQTT broker is a server that receives all messages from the
clients and then routes the messages to the appropriate destination clients.
An MQTT client is any device (from a micro controller up to a fully-fledged
6 - Made many improvements to the TDnsQuery component, to make it easier to use in other components and applications, adding synchronous methods and support for trying lists of DNS servers where one or more don't respond. It is also easier to access the arrays of different DNS answers. Added AAAALookup method for IPv6, similar to ALookup, sets array of IPv6 addresses. Allow lookup using multiple DNS servers if one or more fail, from a supplied list or internal public server list including Google, Cloudfare, OpenDNS and others, only works with the new sync methods. Added OnLogEvent primarily for debugging multiple requests and servers, may mostly be ignored.
7 - Added new components TIcsDomainNameCache and TIcsDomNameCacheHttps designed to simplify forward domain name and reverse IP address lookup in applications, to avoid needing use of TWSocket or TDnsQuery components often several to support parallel lookups. There are synchronous methods that wait until a response is received and asynchronous methods that return immediately with an event called when the response is available. Lookups may be performed using uses winsock so results come from the operating system cache, using the TDnsQuery component to make requests to specific servers or public DNS servers, or using DNS over HTTPs for secure lookups. Up to 100 parallel lookups are supported although defaults to five, with waiting lookups queued and performed first in, first out. TIcsDomainNameCache is mainly for use with diagnostic components but also for servers logging remote access. Used in the OverbyteIcsNetTools, OverbyteIcsNetMon, OverbyteIcsBatchDnsLookup, OverbyteIcsDDWebService and OverbyteIcsSslMultiWebServ sample applications. Could potentially be used in other high level applications to avoid using the operating system DNS cache.
8 - ICS servers and clients can now use SSL/TLS certificates from the Windows Store instead of PEM and PFX disk files, with some limitations. PEM and PFX ICS has long being able to read certificates from the Windows Store, but reading some private keys failed due to Windows APIs issues, for which a workaround has now been implemented so LoadFromStore is now able to read EC keys as well as RSA. TMsX509List allows all certificates and private keys in a store to be loaded, with the new FindBest method finding a certificate matching a search string, checking common name, part friendly name or any alternate domain name, selecting the one with the latest expiry if more than one. If the certificate name is a wildcard (*), this will be matched with any first node For clients, this is illustrated in the sample OverbyteIcsHttpRestTst to select a client certificate, for servers, IcsHosts has a new property SslLoadSource which can be set to CertWinStoreUser or CertWinStoreMachine. with former causing the store to be searched for the host name. Note the application will need administrator rights to load certificates from the Local Machine store, it will also not read keys from external hardware devices, and probably not in Trusted Platform Modules either. The sample OverbyteIcsPemTool can be used to view, select and extract certificates from the Windows Store.
9 - There are a lot of general stability improvements in various low level components, protecting inherited class destroy methods from exceptions at higher levels to avoid memory leaks, such as closing a socket left open before destroy, particularly with Win64 applications. Generally free objects instead of destroying them. IcsX509VerifyErrorToStr now checks OpenSSL is loaded to avoid an exception when it's called. If neither OpenSSL DLL can be found, report both names instead of just oldest. Ensure OpenSSL is loaded before using hash functions.
10 - For ICS servers, increased the default IcsHosts security level to sslSrvSecHigh for TLS/1.2 minimum, with certificate key size 2,049 bits and SHA-256 digest. Added CliCertMethod to IcsHosts to allow specific hosts to request a client SSL/TLS certificate rather than all hosts.
11 - HTTP clients have a number of improvements. Some servers allow the GET and DELETE requests to have content similarly to PUT so allow this if new Options httpoGetContent is specified. Beware to set SendStream to nil if no content is intended. Try to prevent the header and content from being sent as separate TCP packets which may confuse some middleware. Always check if SendStream exists before accessing it. Added OnSyncRequestWait event called while waiting for sync operations to finish, so it can be stopped before timeout expires. Improved EHttpException messages with more detail. In TSslHttpRest, when a client certificate is requested, check it has a private key and log some information about it, better logging if no certificate. The sample OverbyteIcsHttpRestTst has a new Client SSL Certificate drop down box with options to load the certificate from a PEM/PFX file, or from the Windows Current User Store or Windows Local Machine Store (admin rights needed). A real application could offer a selection of which certificate to use as browsers do. Added ics-client-test.pem as default Client SSL Certificate, for testing against ICS servers.
12 - FTP clients have a number of improvements. Added NoopAsync method sends NOOP no operation command to try and keep the control connection alive during long transfers, note there is no sync version and the response is ignored (but logged). Beware some older FTP servers may treat NOOP as illegal and fail the transfer (including ICS FTP server V8.70 and earlier). NOOP is sent by TIcsHttpMulti defaulting to 10 minutes. Without this change, FTP transfers to some public servers are failing after two or more hours due to the control connection having been closed by a router or firewall somewhere for inactivity, now tested OK with 50GB uploads (VM images). Added UTF8_ON and UTF8_ONAsync commands as an alternate to using Opts command with an argument. Increased sync timeout to 30 seconds and make sure reset with Progress more often so requests don't timeout. Fixed unicode compiler bug parsing Cmlsd/XCmlsd command response. Don't compress iso, pdf, vhd, vhdx files by default. Disable MD5/CRC32 by default, connections are now reliable and they are slow.
13 - The FTP server has a fix to allow the NOOP keep-alive command to be sent while a file transfer command is being processed without causing it to fail. Added exception handling generating directory listings due to strange directory inputs by hackers causing a problem. Give msgNoPortPsv response if PORT, EPRT, EPSV or PASSIVE command has not been sent when a directory command requiring a data channel is received.
14 - When ordering SSL/TLS certificates from Let's Encrypt, the local web server can now listen on both IPv4 and IPv6 addresses for domains with both. Removed OAuth2 authentication code from the component and get the tokens using and event. Simplified checking DNS challenges with the new TDnsQuery sync methods. Added MsCertLoc property to specify which Windows Certificate Store to save certificates if OutFmtWinStore is specified.
15 - When sending SMTP HTML emails, the EMailImages property may be used to add streams as well as files, instead of ImageStream and StreamArray which never worked properly. Use EMailImages.AddObject(filename, TStream) where file name will be used instead of opening the file (the file need not exist).
16 - The proxy server component has a new OnSrvSslHandshakeDone event called when new client connects to proxy server allowing application to check for a client certificate and abort the connection, needs CliCertMethod to be set in IcsHosts to sslCliCertRequire or sslCliCertOption, so a client certificate is requested. Fixed auto certificate ordering reading well-known file on unicode compilers.
17 - OverbyteIcsWSocket includes various new utility functions. WSocketSockAddrToStr converts TSockAddrIn6 with IPv4 or PIv6 address to a string, WSocketIPAddrToSocAddr convert a string IPv4 or IPv6 address into TSockAddrIn6, WSocketFamilyToAF to find family for Windows APIs from TSocketFamily, WSocketIPv6Same to compare two TIcsIPv6Address. Added an overloaded ReceiveTB that returns a TByte instead of the received size, it also no longer fails if the buffer is not initialised.
18 - OverbyteIcsUtils corrects RFC3339_DateToStr to add colon to time zone, RFC3359 requires +00:00, ISO also accepts +0000. Added StringToUtf8TB convert string to TBytes, IcsTextOnStart case insensitive text at start of line, and IcsTBytesToString to convert TBytes to unicode string. IcsWcToMb and IcsMbToWc now use cross platform RTL functions instead of OverbyteIcsIconv and USE_ICONV which have been removed. IcsIconvNameFromCodePage is now POSIX instead of USE_ICONV.
19 - Added ICS Internet Packet Monitoring Components which display internet packets using raw sockets or Npcap NDIS driver, similarly to Wireshark. They are based on Magenta Systems Internet Packet Monitoring Components but updated with IPv6 and new filtering by protocol and IP address to restrict the amount of data being captured. TIcsMonSocket in OverbyteIcsMonSock provides internet packet monitoring using raw sockets. TIcsMonPcap in OverbyteIcsMonPcap provides internet monitoring using the Npcap NDIS driver. The class TIcsMonFilterClass filters captured packets. There is a new sample OverbyteIcsNetMon that is a simplified Wireshark, displaying captured packets or just totaling traffic. Packets may be captured to a textual log for easy saving or to a grid for improved display including examining each packet separately. Filters include all local IPs including broadcast and multicast, or specific protocols or services, in all case either accepting or rejecting packets according to the filter settings, dynamically during capture. This makes it easy to ignore a lot of local LAN traffic from appliances that can obscure higher level traffic.
20 - Added Internet Protocol Helper Component for Windows, updated from the Magenta Systems units with full IPv6 support and new components. TIcsIpChanges in OverbyteIcsIpHlpApi monitors IP address changes and calls an event for new IPs configured or old ones removed, useful for servers where the listening address suddenly disappears. TIcsNeighbDevices in OverbyteIcsIpHlpApi builds a historic LAN MAC device and IPv4 and IPv6 address table using ARP,neighbourhood and IP range scanning with reverse host lookup. Both are tested with sample OverbyteIcsNetTools. There are many new IpHlp functions including IpHlpAdaptersInfo, IpHlpNetworkParams, IpHlpConnsTable, IpHlpTCPStatistics, IpHlpIfTable2, IpHlpAdaptersAddr, IpHlpIpAddrTable, IpHlpIpNeighbTable, IpHlpIPForwardTable, IpHlpGetRouteProtocol and IpHlpIpPathTable, all of which return various Windows networking tables and information that is useful for diagnosing network problems, all illustrated by the sample OverbyteIcsNetTools. For completeness, the sample also includes other ICS components, for Whois, NsLook, Ping and Trace Route. Other new functions include IpHlpGetDnsServers to get a list of DNS servers for this PC, IcsGetMacVendor which uses the Organizationally Unique Network Interface Identifier nmap-mac-prefixes.txt file to get the MAC adaptor vendor, very useful for identifying strange IoT devices on a LAN.
21 - Updated the OpenSSL DLLs included with ICS to 1.1.1v and 3.1.2. Updated all the certificate root stores. Also supports 3.0.10 which can be downloaded from the wiki pages.
22 - Added two new FMX samples, IcsHttpRestTstFmx and IcsSslMultiWebServ, converted from the VCL versions, rather messy since no proper TRadioGroup in FMX so changed to TlistBox without a caption, TListView exists in FMX but without columns so they become TStringGrid, TGrid would be better except it uses virtual data (which the help fails to mention) which needs a lot of rewriting. A few more FMX samples using new ICS components will be added for the next release
Changes in ICS V8.70 include:
1 - V8.70 has various minor improvements providing better compatibility with modern compilers such as more unicode overloads to avoid ANSI string warnings and casts, and more use of TBytes to avoid ANSI strings. Updated various samples to use TIcsRestEmail to support OAuth2 authentication for GMail and Outlook that no longer allow old authentication protocols.
2 - The TIcsFileCopy, TIcsFtpMulti and TIcsHttpMulti file transfer components now support file zipping and unzipping using System.Zip in recent Delphi compilers, instead of the obsolete VclZip which is no longer available. Before a file copy or FTP upload, files may be automatically zipped, useful for large log files, after a file copy, FTP or HTTP download, files may be unzipped in various ways. The W versions of the components for non-unicode compilers still support VclZip.
3 - Added support to TIcsFileCopy to copy file names longer than 259 characters by adding \\?\ to the start of long names passed to Windows APIs, if supported by the disk file system, unicode APIs only. Fixed a problem deleting empty directories after copying. Fixed a problem with BuildDirList2 with COMPILER16_UP where SearchRec.TimeStamp returns a UTC timestamp instead of local time, unlike SearchRec.Time with returns local time, so ignore it. This caused problems with all file transfer components that could not check if old files were unchanged correctly.
4 - The OverbyteIcsXferTst sample has a new tabs, 'Single File Copy' to test the CopyOneFile method and 'Zip/Unzipping Files' to test zipping and unzipping that has always been supported by the components but not this demo.
5 - Allow content compression for HTTP and FTP using System.Zlib in newer versions of Delphi instead of the OverbyteIcsZLibObj unit to avoid duplication. Only Delphi 11.1 and later have the same ZLIB 1.2.12 as ICS, so will automatically used System.Zip. If older ZLIB versions are acceptable in older versions of Delphi, the USE_DELPHI_ZLIB define can be set. Also simplified use of ZLIB so higher level units only need OverbyteIcsZlibHigh which is the only unit now checking defines to use OverbyteIcsZLibObj, OverbyteIcsZLibDll or System.Zip. Beware a new version of OverbyteIcsDefs.inc is required to allow ZLIB to work correctly, otherwise it will default to using the DLL which is unlikely to be available, it is not in the distribution. So either install the new inc file and customise it, or copy the ZLIB changes to your own inc file.
6 - In TWsocket, added ReceiveTB(var Data : TBytes; MaxLen : Integer =
-1): Integer; where MaxLen is optional, to receive TCP data into a TBytes
dynamic array of bytes. Also ReceiveFromTB and ReceiveFrom6TB for UDP
datagrams. The last release added similar SendTB functions, so buffer
pointers and ANSI strings can now be avoided. Added new ComponentOptions
wsoNoSendException which stops an exception being raised if Send is called
when the socket is not connected, which can happen if the connection drops
unexpectedly. Improved Unicode compatibility by adding UnicodeString
overloads of various low level AnsiString functions for getting and setting
IPs, ports and hosts
7 - Added UTF-8 support to TIcsIpStrmLog, to convert received lines from UTF-8 to Unicode with unicode compilers (as String) and converts sent data to UTF-8. Changed FRxBuffer to TBytes, use SendTB and ReceiveTB methods with TBytes. The OverbyteIcsIpStmLogTst sample has a new 'Use UTF-8' tick box so the component sends and receives UTF-8 instead of ANSI, and 'Test Unicode Data' to create some unicode characters for testing.
8 - Updated OpenSSL to 3.0.7 and 1.1.1s. OpenSSL 3.0.6 was withdrawn shortly after release, we never distributed it.
9 - Fixed bug in V8.65 where FloattoStr could create Json with comma decimal point instead of a period with some languages.
10 - In OverbyteIcsSslHttpOAuth, added an OAuth2 and Rest Email Microsoft User Authority property to access different user authorities, defaults to 'consumers' but can be changed to 'common' or an Azure Active Directory tenant GUID for corporate accounts. Also added IcsLoadRestEmailFromIni to load TIcsRestEmail secrets and tokens from an INI file. Ideally these settings should be encrypted!!
11 - Added TIcsRestEmail to support OAuth2 authentication to the OverbyteIcsSslMultiWebServ, OverbyteIcsSslMultiFtpServ and OverbyteIcsDDWebService samples, since GMail and Outlook that no longer allow old authentication protocols. Need a new [RestEmail] section in INI file with OAuth2 account client id and secret and refresh token (see top of OverbyteIcsSslHttpOAuth.pas unit). The refresh token can be obtained using the OverbyteMailQuTst sample with the same client account.
12 - In TIcsMailQueue, don't use synchronize for logging in thread if component is multithreaded, it blocks the queue. Fixed a memory leak. In the OverbyteIcsMailQuTst sample, set SMPT UTF-8 charset so headers get inline MIME encoded if necessary, rather than being converted to ANSI. Note body lines are not converted in this sample. Added Microsoft User Authority property to access different user authorities.
13 - In the OverbyteIcsSslMailSnd sample, added header character set and body line wrapping and encoding from non-SSL sample. Note the 'Use Mail Message' check box means you pass the entire body in MailMessage property instead of using the OnGetData event to send one line at a time, this will also word wrap long lines and encode body content.
14 - In the TIcsInetAlive component, added a new method AliveMethEither so internet alive checking works if either ping or HTTP works, instead of one or the other.
15 - Added a new utility function IcsBuiltWithEx that returns the compiler version for display, ie 'Delphi 11.2 Win32'. Added to several samples to ease testing.
16 - In OverbyteIcsLIBEAY, corrected the names of Win64 providers for YuOpenSSL.
Changes in ICS V8.69 include:
1 - V8.69 has a lot of minor improvements, more HTTP client and server features, better built-in authentication, and added built in file uploading to the REST client component. Also continuing improving all the HTTP samples so they now hopefully test all the features of the ICS HTTP server, specifically adding POST and PUT file uploading, and a new client login window for interactive authentication with servers. Also added OCSP (Online Certificate Status Protocol) to ICS, which is used to check SSL/TLS certificates are legitimately issued and not revoked.
2 - In the HTTP client, added new methods RequestAsync and RequestSync which start a specified request by parameter, useful to repeat the last request from an event (such as should be trust the certificate and repeat). When using a proxy, make sure Path is not blank which will break proxies, change to / as for non-proxy requests. GetRequestDoneErrorStr now handles TWsocket errors. Fixed NTLM authentication which got broken in V8.61. Added a new THttpAuthType of httpAuthDigest2 for Digest with the modern SHA-256 algorithm instead of MD5 used by httpAuthDigest, only with USE_SSL. Added new property WWWAuthInfos array filled after 401/407 failure by parsing AuthorizationRequest headers for AuthType and Realm that may be presented to select an authentication option, get login and repeat request. Remove # fragment anchor rom the URL unless the new Option httpoAllowAnchor is set.
3 - In the HTTP REST client, added built in file uploading using POST or PUT. The file name is specified in HttpUploadFile using type HttpUploadStrat ofHttpUploadSimple with parameters in the URL or HttpUploadMIME for multipart with parameters in the first MIME part. Note the applications needs to supply parameters like FileName so the server knows what to do with the file. This may be tested against the ICS web server samples. Upon request completion, ReasonPhrase now also has status, so OK becomes 200 OK, etc. Previously the SslRevocation property was only effective when checking the windows certificate store, now it also works with bundle files using the new TOcspHttp component and OCSP stapling if available.
4 - In the HTTP server component, now converting FLastModified to UTC/GMT time for the response header. Added new authentication type atDigestSha2 with a SHA256 hash instead of MD5 with atDigest only supported with USE_SSL. Note: Mozilla Firefox supports Digest SHA-256, Chrome and Edge do not. Added a new client connection OnHttpRespHdr event to allow response headers to be logged, previously only request headers could be logged. Cleaned up AnswerStream functions so separate lines don't go into send buffer. Added OcspSrvStapling property which should be set to enables OCSP checks and stapling only with AUTO_X509_CERTS define since it adds extra HTTP client code. A revoked certificate will be auto ordered.
5 - In TWSocket, added new method SendTB(const Data: TBytes; Len: Integer=-1) where Len is optional, also similar SendToTB and SendToTB6 with TBytes, as an alternative to casting such types to use Send(). Restored the TlsExtension_cb callback for client debugging since ClientHelloCallback only works with servers.
6 - In TSslWSocketServer added OCSP (Online Certificate Status Protocol) support with IcsHosts using the TOcspHttp component to confirm server SSL/TLS certificates are legitimate and not revoked for security reasons. The certificate OCSP response is also stapled to the initial SSL/TLS HELO handshake and sent to the client to avoid it needing to lookup OCSP using HTTP itself. OCSP responses are cached and saved to a file for reloading later, but are refreshed every time the certificate is validated, at least once a day. The new server property OcspSrvStapling enables OCSP checks and stapling only with AUTO_X509_CERTS define since it adds extra HTTP client code. A revoked certificate will be auto ordered. OCSP checking is done in LoadOneCert and the stapled response sent in TriggerSslServerName when checking SNI. The same OCSP support is available in all ICS servers that use IcsHosts, including FTP, HTTP and TIcsIpStrmLog.
7 - All three main HTTP SSL client sample applications FrameBrowserIcs, OverbyteIcsHttpsTst and OverbyteIcsHttpRestTst now support interactive authentication with a new Login window that displays the different methods the server will accept (from the new property WWWAuthInfos array) allowing one to be selected from Basic, Digest MD5, Digest SHA256 and NTLM logins. This window is displayed after an 401 error and the request then repeated.
8 - The OverbyteIcsHttpsTst SSL sample now has all the missing features from the non-SSL samples, Content Encoding Gzip tick box to support compression, persistent cookie support, POST/PUT support to either send simple data or upload files in various ways (from OverbyteIcsHttpPost1 sample), may be tested against the ICS web server samples.
9 - The OverbyteIcsDDWebService SSL web server sample now builds on unicode compilers. Added authentication for POST requests and new 'Password protected page (POST)' button on the demo menu to test authentication using POST. Fixed web logging to log correct multiple listener. Builds on unicode compilers. Added Digest SHA-256 authentication page DemoDigest2Auth.html and DemoDigestsAll.html that does both digests. DemoAuthAll.html no longer does NTLM, use the separate page. Now displays server response headers if box ticked, only displayed request headers before. File Upload Form and Email Form pages now work without exceptions. The OverbyteIcsSslMultiWebServ SSL web sample has similar new authentication features and logging fixes.
10 - Updated OpenSSL to 3.0.3 and ZLIB (HTTP compression) to 1.2.12, sorry for long delay in updating zlib, now including some important bug fixes, although never saw any issues with the minimal use ICS makes of it. Updated the various ICS CA Trusted Stores.
11 - The OverbyteIcsPemtool SSL sample now does OCSP checks when examining certificate files. There is also a new Test Host Certificates tab that tests SSL/TLS handshake and certificates (using TIcsIpStrmLog) for a list of host names and ports, building a list of host and intermediate certificates that may be further checked and saved as files or added to an intermediate bundle. This new tester should prove useful for debugging sites that return SSL or certificate errors.
More detailed release notes are at ICS 8.69 Release Notes
Changes in ICS V8.68 include:
1 - V8.68 is a minor release, mainly HTTP client and server improvements including new request and response headers to assist browser caching and conditional requests, improved error reporting to help diagnose failed HTTP requests, and improvements in the HTTP REST component allowing it to save files including resuming failed downloads and download files of any size. There are minor fixes in various components, and updates to installing on MacOS and C++ Builder. There is also a new File Clean-Up demo ideal for deleting the old log files that many applications (including some ICS samples) leave on systems.
2 - The HTTP client TSslHttpCli now keeps the Etag response header as ResponseEtag, allowing applications to save it with the page content, and when requesting a refresh to add the request method ReqIfNoneMatch or ReqIfMatch to avoid downloading the page again. Made RequestDoneError available as property so it can be accessed after Sync requests, and added RequestDoneErrorStr property that returns a literal error. Improved RequestDoneError to give more information than a simple abort, added httperrOutOfMemory and httperrBgException which happen while receiving and processing data in the OnDocData event and httperrSslHandShake. Improved AbortComponent so ReasonPhrase now reports the exception that caused it, such as out of memory which previously needed a BgException event handler. For range downloads, added the ContentIfRange request header which can send an RFC1123 date or Etag so a partial download only happens if the file is unchanged.
3 - Previously, the HTTPS REST client TSslHttpRest always downloaded content to a TMemoryStream with content size being limited to MaxBodySize (default 100 MByte), and generally restricted by memory to less than 250 MByte. To remove this limitation, added the HttpMemStrategy property with THttpMemStrategy on how to handle downloads: HttpStratMem only TMemoryStream; HttpStratTemp uses a work file in the system temporary directory for sizes larger than MaxBodySize; HttpStratFile always writes a named file HttpDownFileName (with .part extension during download); HttpStratResume is similar to HttpStratFile but supports resume of failed partial downloads (with .http extension for resume information). Property ResumeMinSize defines the minimum sized partial file that should be resumed, rather than start again (default 64K). Note MaxBodySize remains the maximum size for ResponseRaw (unicode string), JSON and XML parsing. Note this is a breaking change since the ResponseStream property is now TStream, so to use the SaveToFile method you will need to be cast as TMemoryStream. Or change to use HttpStratFile method with HttpDownFileName. Also note, ResponseStream remains open after request completes and may occupy a lot of memory or leave file open (read only), also ResponseRaw string, so use ClearResp method when no longer needed. The component now logs a better error if the response does not contain Json. Added ShowProgress property that causes download information to be sent to the OnHttpRestProg event using LogOption=loProgress, showing progress in KBytes of KBytes updated every ProgIntSecs seconds, default two, expected to be displayed as a caption. The OverbyteIcsHttpRestTst.dpr sample has fields to select a Download File Name when saving a file and for Memory Strategy, download progress is also displayed.
4 - The HTTP server TSslHttpServer now supports the If-Range, If-Match and If-None-Match request headers for conditional pages using Etag or last modified date, and sends 304 not modified for matches, to help with caching. Added an ETag header to responses in AnswerStream and AnswerPage when we can create one from a file modification date and size (base64 CRC32), or if the EntityTag property is specified in the client onGetDocument event before using hgSendDoc or hgSendStream, perhaps a CRC32 of the entire content from a cache. The 304 not modified response now includes more recommended headers. Also add a Date: header to AnswerStream and AnswerPage responses to help with caching. Note, if applications already add Date: or Etag: to CustomHeaders or to an AnswerStream should remove them to avoid duplication.
5 - The Multi HTTPS client TIcsHttpMulti has been updated to a fix a problem that meant large files failed download with only an abort error if too large for TMemoryStream, now downloaded to TFileStream with .part extension and renamed up successful completion. If a partial file downloads, don't delete it if KeepPartDown=True. Added logging of RequestDoneError on failure and BgException. Correctly download files using chunked coding without known size, note these show with size -1 since not known until complete. These fixes were triggered by failed HTTP 300 MByte downloads from a new HD CCTV camera that said abort with no mention of out of memory errors, thus a lot of debugging and ICS changes to improve matters.
6 - The previous ICS release added support for the OpenSSL 3.0 release. Now it's been available for three months it has been added to the main distribution, the samples SslInternet directory now has both OpenSSL 1.1.1m and 3.0.1, ICS will try and load OpenSSL 3.0 first, then 1.1.1 if not found, unless the global variable GSSLEAY_DLL_IgnoreNew is set true before OpenSSL is loaded. Likewise GSSLEAY_DLL_IgnoreOld may be set true to ignore 1.1.1 and fail unless 3.0 is available. ICS now supports YuOpenSSL 3.0 and 1.1.1 versions as commercial DCUs allowing applications to be used with OpenSSL without needing separate DLLs, except it still loads providers like legacy.dll as DLLs, not a DCU.
7 - In TIcsWndControl, BgException now passes the exception to the AbortComponent virtual handler and ExceptAbortProc method so higher level components can report errors better, such as out of memory without needing an BgException handler. Note requires similar change to all units that override AbortComponent or use ExceptAbortProc.
8 - The TX509Base component now supports OpenSSL 3.0 for Load/SaveP7BFile. ValidateCertChain no longer reports an error for the expired 'DST Root CA X3' CA root, since some platforms accept expired roots. Improved OpenSSL error handling to say 'No error returned' instead of error:00000000:lib(0):func(0):reason(0). TWSockey now clears LastError in Listen and Connect methods to avoid false errors later. Improved source description for some fatal background errors.
9 - In TSslX509Certs, added property KeepOldCA set true to keep the old Let's Encrypt intermediate for the expired DST Root CA X3 root in bundles to support old Android releases. Keeping it may prevent some clients verifying the chain and SslLabs testing gives a chain warning.
10 - In TIcsFileCopy, the DeleteFiles method now supports a new Zipped property so files are zipped with .zip extension before being deleted, useful for cleaning up old logs to save space (requires VCLZip). The OverbyteIcsXferTst,dpr sample has a new File Clean-Up tab to test the multiple DeleteFiles method, it allow files older than x days or a specific date range to be archived/zipped (to save space) or deleted, ideal for deleting the old log files that many applications (including some ICS samples) leave on systems.
11 - In the FTP client TSslFtpClient, using the PORT command to set Active mode now prevents other FTP clients sharing the same port number and address, usually only a problem with a small port pool (which is not recommended). For an Abort, LastResponse now reports the exception that caused it, such as out of memory which previously needed a BgException event handler.
12 - In the Multi FTP client TIcsFtpMulti, improved TLS session caching by ignoring the connection port so the data sessions can use the control session.
13 - Updated the trusted root certificate bundle files, lots of changes from Microsoft since June, Google is now issuing it's own certificates. Updated the build-in sslRootCACertsBundle, few gone, now total 59 certificates commonly used.
14 - ICS should now build with C++ for RAD Studio 10.4 and 11.0, fixed the 11.0 packages and various Windows API related units, including for Win64. The Platform FMX samples now build with RAD Studio 11.0 where some FMX properties have changed, not sure if they'll still build with older versions. Building samples no longer needs the source search path to have \Include, this has been added to all the sample units that use OverbyteIcsDefs.inc.
More detailed release notes are at ICS 8.68 Release Notes
Detailed ICS 8.67 Release
Now part of ICS V8.60 and later, as TIcsMailQueue
Magenta Systems Mail Queue Component
Magenta Systems Mail Queue Component has two main benefits over a simple TSslSmtpCli component: it supports extended retries over many hours or days, and supports multiple SMTP relay servers or looks-up MX servers, while alleviating the need for the application to handle retries. The component also allows HTML mail to be sent using SSL, something THtmlSmtpCli does not currently support. Mail is queued to disk, so retries will continue if the application is restarted.
TMagMailQueue is designed to prepare, queue and send email. Preparing the email is done using the ICS THtmlSmtpCli component so it may be plain text or HTML email with one or more file attachments.
Once the mail properties in QuHtmlSmtp have been specified, it is queued using the QueueMail method which saves it to an EML spool file.
The component runs a thread which checks the queue for new EML spool files, and attempts to forward them to one or more SMTP Mail Servers using TSslSmtpCli, optionally with SSL. If mail delivery succeeds, the spool file may be deleted or moved to an archive folder. If mail delivery fails, the spool file remains in the queue and further attempts are made separated by the times in minutes listed in the RetryList list. If all delivery attempts fail, the spool file may be deleted or moved to a badmail folder.
Note that some email servers support grey listing and reject the first email attempt from a new sender but allow a retry 10 or 15 minutes later, something that is very effective in blocking spam emails (since they don't usually retry).
If multiple mail servers are specified, delivery is attempted once using each server, for each retry attempt. Each mail server is specified as TMailServer and there is no limit to the total.
Each time the queue is updated or a delivery attempt made, the queue is saved to file in the control folder, so the component may be stopped and restarted with failed attempts continuing.
The EML spool files are compatible with those created by many Microsoft email applications such as CDO, and the AddtoQueue method can also be used to queue existing EML files with the queue details specified in MailQuItem.
Note, this component is intended for sending low volume email from individual Delphi applications, with more flexibility than a simple TSslSmtpCli component. For use as a heavy duty SMTP server, queue processing could be improved to avoid moving records around as much or saving them to disk as often, and mail bodies could be read as required from disk instead of being read entirely to memory first. A mail pickup folder could be added which is scanned for new EML files.
Files and Folders Used
The TMagMailQueue component heavily uses disk files, in different sub-directories within the mail root directory specified in property MailQuDir, these are:
If logging of sent email is specified, the default file name FileQuSent property is MailQuSent-yyyymmdd.log inb CSV format similar to MailQuItems.Hdr.
A demo application mailqudemo.exe illustrates simple email queuing. The zip contains the EXE demo and required SSL files.
18th January 2011 - 1.0 - first public release. Not yet tested with Delphi 2009 or later.
2nd March 2011 - 1.2 - automatically create mailqueue directory in demo application, removed missing uses statement. Support queuing mail with OwnHeaders bypassing htmlmail. Log event definition changed.
11th August 2011 - 1.2 - updates subroutines for Win64 support, removed one unneeded unit from uses.
5th Oct 2011 - 1.3 - Debug logging works properly Don't retry emails that fail too large for server (error 552)
11th Sept 2012 - 1.4 - ICS V8, IPv6
23rd March 2013 - 1.5 - Added Mail Server SocketFamily and LocalAddr6 for IPv6
10th Dec 2014 - 1.6 - Better SSL handshake reporting
27th Oct 2015 - 2.0 - requires ICS V8.19 October 2015 or later.
7th July 2016 - 2.1 - requires ICS V8.30 July 2016 or later.
1st December 2016 - 2.2 - requires ICS V8.39 November 2016 or later.
6th March 2017 - 2.3 - requires ICS V8.43 March 2017 or later.
11 Mar 2017 - 2-4 - Added WaitSend to wait until everything sent.
22th Jun 2018 - 2.5 - requires ICS V8.55 20 June 2018 or later.
26th November 2018 - 2.5 - tested with ICS 8.58
Now part of ICS V8.60 and later.
Now part of ICS V8.60 and later, as TIcsIpStrmLog
Magenta Systems IP Log Streaming Component
TMagIpLog is designed for IP stream logging, using TCP Client, TCP Server, UDP Client or UDP Server protocols, sending simple text lines across a network so they may be displayed or written to disk remotely. The component allows two way communication with TCP and UDP, so may also be used for simple protocols such as communication between two applications. The component supports multiple client sockets so may be used to send data to two or more different remote servers at the same time.
For TCP and UDP clients, the component will optionally ping the remote computer first before opening an IP connection to allow faster failure retries and some confirmation that UDP may work. TCP client provides repeated connection retry attempts, including re-establishing a lost connection. UDP client will optionally keep pinging the remote during a connection to ensure it's still there. UDP server sends data to the IP address and port from which it last received data. TCP server supports multiple remote clients connecting. Received data is parsed for various line endings optionally removing control characters and triggering an event for a received line. The only other two events are optional, one for state changed when starting and stopping, the second offering progress information and errors.
The component supports both IPv4 and IPv6, host name lookup for TCP and UDP Client, and SSL connections for TCP Client and TCP Server, including remote server certificate checking using either a local PEM bundle root file or the Windows Certificate Store.
A demo application testiplog.exe illustrates use of TMagIpLog as a TCP or UDP client or server, and both in the same program sending data locally. The same component may be used in a client or server application, to send or receive.
The Magenta Systems ComCap application may also be used to capture IP streams to files or a database.
1 - Drop the component onto a form (or create it in code, see testiplog.exe).
2 - Specify LogProtocol as one of logprotUdpClient, logprotUdpServer, logprotTcpServer, logprotTcpClient.
3 - For client protocols, specify RemoteHost (name or IP address) and RemoteIpPort, CheckPing true if ping to be used, RetryAttempts to non-zero if continual retries not needed, RetryWaitSecs for delay between retries .
4 - For server protocols, LocalIpAddress is 0.0.0.0 to listen on all local addresses, LocalIpPort must be non-zero.
5 - For sending data, AddCRLF to false if line already have terminating characters, UdpNoCRLF to false if UDP should send CRLF.
6 - For receiving data, LineEndType to one of lineendCR, lineendLF, lineendCustom (set in hex in CustomLineEnd) or lineendPacket (for UDP), then MaxLineLen if a line should be returned before lineend is found, normally non-ASCII characters are removed, set StripControls to false if they should be replaced by spaces, RawData to true if CR, LF, FF and control characters should not be removed.
7 - Assign onLogRecvEvent if data is to be received, onLogChangeEvent if tracking of start and stop is needed, onLogProgEvent if progress information is needed for logging.
8 - Call StartLogging. The LogChangeEvent and LogProgEvent will trigger when LogState changes to logstateOK when data may be sent.
9 - To send a line, if function GetAnyStateOK is true, call SendLogLine. MaxSendBuffer specifies the amount of data that can be buffered otherwise SendLogLine will fail.
10 - Received data will trigger LogRecvEvent once per line.
11 - Call StopLogging to stop. Buffered data may continue to be sent after close, keep calling CheckStopped until true when it's really finished and component may be destroyed.
12 - To send an unlimited size stream, create a stream in the application with TBufferedFileStream or TFileStream, and pass it to SendStream. LogState changes to logstateOKStream while it's being sent, then back to logstateOK as it finishes, the application should then free the stream.
13 - There is no specific handling for receiving a stream, textual data will be handled according to the normal line end properties, and can be saved to another stream in LogRecvEvent. Binary data is more problematic, set RawData to true and MaxLineLen to get a buffer load at a time, but the last buffer load will need to be extracted with GetPartialLine using a timeout, this is called automatically when the connection is closed.
14 - To send to multiple clients, set MaxSockets to the number needed, then use the function SetRemotes to specify the remote host and port for each socket number, base 0. The events all return Socnr to indicate which socket. MaxSockets also specifies how many remote clients can connect to TCP Server, but note that Socnr is dynamic and changes as remote clients come and go.
15 - To support SSL on TCP/IP client or server, drop an TSslContext component on the form, assign it to the LogSslContext property and set the ForceSsl property to true. For better performance, set LogSslSessCache to a TSslAvlSessionCache component.
16 - For SSL TCP Server, the SslContext component must have the SslCertFile and SslPrivKeyFile properties set to the file names of an SSL certificate and Private Key PEM files respectively, and SslCipherList set to sCipherMozillaSrvBack for strong but backward compatiblle cipher support. The component includes sample self signed certificate and password files iplog-cert.pem and iplog-prvkey.pem, and you can create your own with the ICS SSL sample application Pemtool, or buy commercial PEM certificates.
17 - For SSL TCP Client, the SslContext component must have the SslCAFile property set to the file name of a PEM root certification authority file containing trusted root certificates. Such a file is supplied with the component RootCaCertsBundle.pem containing various root certificates covering most major registries. SslContext SslCipherList can be left as the default to allow connection to any server. The LogSslVerMethod property can be logSslVerNone to skip certificate verification, logSslVerBundle to check using the CA bundle file or logSslVerWinStore to check using the Windows certificate store (a little slower, bur maybe more certificates). To check if certificates have been revoked set LogSslRevocation to true, beware this needs public internet access and can be very slow or fail. LogSslReportChain set to true reports certificate details checked.
18th August 2007 - 1.1 - using OverbyteIcsFtpSrvT instead of OverbyteIcsLibrary, UDP receive packets may be from multiple hosts, always keep IP.
5th August 2008 - 1.2 - made compatible with ICS V7 and Delphi 2009. Note only supports ANSI with Delphi 2009.
20th August 2009 - 1.3 - fixed problem with MaxSockets being reported as closed in the event when only one was open, tested with Delphi 2010.
9th August 2010 - 1.4 - removed cast warnings with Delphi 2009 and later
22nd Sept 2011 - 1.5 - added SndBufSize and RcvBufSize to increase buffer sizes and speed
11th Sept 2012 - 1.6 - better error for too many clients with server added CurSockets property for current number of server sockets
7th July 2014 - 2.0 - now only ICS 8 and later, using new ICS ping.
13th July 2015 - 2.2 - requires ICS V8.18 June 2015 or later.
23rd Oct 2015 - 2.3 - requires ICS V8.19 October 2015 or later.
8th July 2016 - 2.4 - requires ICS V8.30 July 2016 or later.
23rd Nov 2016 - 2.5 - requires ICS V8.39 November 2016 or later.
7th March 2017 - 2.6 - requires ICS V8.43 March 2017 or later.
22nd June 2018 - 2.7 - requires ICS V8.55 20 June 2018 or later.
14th December 2018 - 2.8 - tested with ICS 8.58
Now part of ICS V8.60 and later. Uses IcsHosts.
Now part of ICS V8.60 and later, as TIcsWhoisCli with a new sample application
Whois Component and Demo
A Whois component and demonstration application. Whois is a protocol to interpret a remote server for information about a domain name or an IP address, and return textual information about 'owner' of the name or address. The demo application interprets the result and will perform a secondary query to another Whois server if necessary.
Now part of ICS V8.60, as TIcsTimeClient and TIcsTimeServer with a new sample application
SNTP Time Server and Client Components
TTimeServ is an updated version of Nathan Anderson's time server component adding SNTP support. TWSTimeClient is an updated version of Chris Barber's time client component adding SNTP support and functions to change the PC UTC time. SNTP provides time correction with fractional seconds, unlike the earlier Time protocol that is round seconds only.
Magenta Systems Ltd, 9 Vincent Road,
Croydon CR0 6ED, United Kingdom