Dimensional Data

What is web scraping? Web Scraping is a technique where a computer program extracts data from human-readable output coming from websites. The web scraping software may directly access the World Wide Web using the Hypertext Transfer Protocol or a web browser. While Web Scraping can be done manually by a software user, the term typically refers to automated processes implemented using a program, bot, or web crawler. It is a form of copying in which specific data is gathered and copied from the web, typically into a central local database, spreadsheet, API, or any format that is more useful for the user, for later retrieval or analysis. First, the app needs to interpret a web page as data Web pages are built using text-based mark-up languages; like HTML and XHTML, and frequently contain rich and useful data in text form. Quite obviously, most web pages are designed for human end-users and not really for ease of automated use. As a result, this can make it a challenging task to build specialized tools and software to facilitate the scraping of any web pages. Delphi plus Python is a powerful combination for web scraping In this tutorial, we’ll build Windows Apps with extensive Web Scraping capabilities by integrating Python’s Web Scraping libraries with Embarcadero’s Delphi, using Python4Delphi (P4D). P4D empowers Python users with Delphi’s award-winning VCL functionalities for Windows which enables us to build native Windows apps 5x faster. This integration enables us to create a modern GUI with Windows 10 looks and responsive controls for our Python Web Scraping applications. Python4Delphi also comes with an extensive range of demos, use cases, and tutorials. We’re going to cover the following… How to use Requests, BeautifulSoup, Instaloader, Snscrape, Tweepy, and Feedparser Python libraries to perform Web Scraping tasks All of them would be integrated with Python4Delphi to create Windows Apps with Web Scraping capabilities. Prerequisites Before we begin to work, download and install the latest Python for your platform. Follow the Python4Delphi installation instructions mentioned here. Alternatively, you can check out the easy instructions found in the Getting Started With Python4Delphi video by Jim McKeeth. Time to get started! First, open and run our Python GUI using project Demo1 from Python4Delphi with RAD Studio. Then insert the script into the lower Memo, click the Execute button, and get the result in the upper Memo. You can find the Demo1 source on GitHub. The behind the scene details of how Delphi manages to run your Python code in this amazing Python GUI can be found at this link. Open Demo01.dproj. How do I Scrape Website’s Data using Python Requests? “Requests” is a simple, yet elegant HTTP library. Requests allow you to execute standard HTTP requests extremely easily. Using this library, you can pass parameters to requests, add headers, receive and process responses, execute authenticated requests. Requests are ready for the demands of building robust and reliable HTTP–speaking applications, for the needs of today. Keep-Alive & Connection Pooling International Domains and URLs Sessions with Cookie Persistence Browser-style TLS/SSL Verification Basic & Digest Authentication Familiar dict–like Cookies Automatic Content Decompression and Decoding Multi-part File Uploads SOCKS Proxy Support Connection Timeouts Streaming Downloads Automatic honoring of .netrc Chunked HTTP Requests After installing Python4Delphi properly, you can get Requests using pip or easy install to your command prompt: […]

Lets take a look at this image below. Let’s try to think about 2 or 3 objects we can see and focus on what calls our attention the most. If we write them down , lets see if Google can guess it right. Google’s cloud-based vision API – making sense of what we see and much more Google Cloud’s Vision API offers powerful pre-trained machine learning models that you can easily use on your desktop and mobile applications through REST or RPC API methods calls. Lets say you want your application to detect objects, locations, activities, animal species, and products. Or maybe you want not only to detect faces but also their emotion expressed on the faces. Or perhaps you have the need to read printed or handwritten text. All of this and much more is possible to be done for free (up to first 1000 units/month per feature) or at very affordable prices and it’s scalable too with no upfront commitments. Object localization The option for “Object Localization” is part of the Vision API that we can use to detect and extract information about multiple objects in an image. For each object detected the following elements are returned: A textual description – what is it, in plain human language? A confidence score – how certain is the API of what it has detected? And normalized vertices [0,1] for the bounding polygon around the object. Where are the objects on the image? Using RAD Studio Delphi to control the Google Cloud Vision API We can use RAD Studio and Delphi to easily setup its REST client library to take advantage of Google Cloud’s Vision API to empower our desktop and mobile applications and if the request is successful, the server returns a 200 OK HTTP status code and the response in JSON format. Our RAD Studio and Delphi applications will be able to either call the API and perform the detection on a local image file by sending the contents of the image file as a base64 encoded string in the body of the request or rather use an image file located in Google Cloud Storage or on the Web without the need to send the contents of the image file in the body of your request. How do I set up the Google Cloud Vision Object Localization API? Make sure you refer to Google Cloud Vision API documentation in the Object Localization section (https://cloud.google.com/vision/docs/object-localizer), but in general terms this is what you need to do on Google’s side: Visit https://cloud.google.com/vision and login with your Gmail account Create or select a Google Cloud Platform (GCP) project Enable the Vision API for that project Enable the Billing for that project Create a API Key credential How do I call Google Vision API Object Localization endpoint? Now all we need to do is to call the API URL via a HTTP POST method passing the request JSON body with type OBJECT_LOCALIZATION and source as the link to the image we want to analyze. One can do that using REST Client libraries available on several programming languages and a quick start guide is available on Google’s documentation (https://cloud.google.com/vision/docs/quickstart-client-libraries). Actually, at the bottom page of the Google Cloud Vision documentation Guide (https://cloud.google.com/vision/docs/object-localizer) there is an option “Try This API” which allows you to post […]

If you’re like me flying is probably a slightly mystical experience. You’re not entirely sure how the plane defies gravity and the job of a pilot seems to involve a bewildering array of switches, dials and incredibly expensive sunglasses. There’s a lot more going on, of course, beyond the cockpit and safety briefings. Modern flights, even in smaller aircraft, involve much more tedious ephemera like equipment reservations, invoices, fees and the relentless recording of flight data. Making the process of flying less trying Belgian developers Micriconsult have poured their talent into a dedicated Android app specially for members of The Noordzee Vliegclub. Using RAD Studio Delphi and the Firemonkey FMX framework they have helped ease and automate some of the tasks of the flying business with some thoughtful and intelligent workflow analysis to create a really useful tool which pilots can simply carry around in their pocket – especially useful for smaller light-commercial and pleasure craft where space is at a premium. Noordzee Vliegclub Website What does the Noordzee Vliegclub app do? The app allows a member from the Noordzee Vliegclub to reserve a plane, to enter the flight data to help fulfil legal obligations as well as keep tabs on the myriad expenses of the modern world of flight. According to the developer, “It also keeps track of all flights and all costs involved, presented as monthly invoices. At any time a member can update its database data. This app allows a member from the Noordzee Vliegclub to reserve a plane and enter the flight data. At any time a member can update its database data. A flight instructor can define a plane as a training plane (preventing members to use it for regular flights), authorize a pilot member to fly on a specific plane type and keep track on the flying hours of each student. Users who are not a member of the Noordzee Vliegclub cannot use this app.” Google Play Noordzee Vliegclub Screenshot Gallery You could really make a difference – use your talent with RAD Studio today and be a high-flyer!