Learn how to find real-time weather data using web scraping and APIs. You can use this fetched data to develop a simple weather application.
Get Current Weather Details of a City Using Web Scraping
Web scraping is the process of extracting data and content from a website. Autonomously fetching data from the web opens up a lot of use cases. But most of this data is in HTML format, which you need to parse and inspect to extract relevant data.
You can extract live weather data of any city using web scraping. Python’s BeautifulSoup library is the go-to library to pull data out of HTML and XML files. You need to install the BeautifulSoup Python library via pip to begin the scraping process. Run the following command in the terminal to install the BeautifulSoup and requests libraries:
After you’ve installed the required libraries, start by importing them in your code:
Next, you need to provide the header details so that the client and the server can pass additional information with an HTTP request or response:
Create a function, find_weather(), to make a query to Google using the requests.get() method. You’ll use a search URL to get a city’s weather, then scrape the meaningful data to get location, temperature, time, and weather description. Then, use BeautifulSoup to parse the received HTML response:
To extract the element IDs, carry out a Google search and inspect the page in your browser using web tools. You need to inspect the element to find the IDs of the HTML element for which you want to extract data:
Next, pass these IDs to the select() method. This method runs a CSS selector against the parsed document and returns all the matching elements. The getText() method extracts the text from the HTML element. The strip() method removes any leading and trailing whitespace characters from the text. Once you’ve extracted a clean value, you can store it in a variable.
Finally, ask the user to input a city and pass it to the find_weather function:
Now, when you run the code, it will prompt you to enter a city name. You must enter a valid city name to get the results or the code will raise an exception.
Get Current Weather Details of a City Using OpenWeatherMap API
OpenWeatherMap is an online service, owned by OpenWeather Ltd. Its API provides global weather data including current weather, forecasts, and past data for any location. The free tier of the OpenWeatherMap API provides current weather data with a limit of 60 calls/minute. You need to create an account on OpenWeatherMap to get your own API key.
Go to OpenWeatherMap’s website and create a free account. After creating the account, you can find your API keys on the My API Keys page. You can use the default API key provided by the OpenWeatherMap or generate one of your own. OpenWeatherMap provides the support to generate as many API keys as needed for your projects.
Now, you’re ready to retrieve the live weather data.
If you provide a valid API key and enter the correct city name, you’ll receive the data from the API in JSON format. Next, you need to convert this JSON format data into a Python object using the json() method to perform further operations. If the city is found, you will have to resolve the dict object (res) to extract the required information.
Develop Weather Application Using the Live Weather Data
Now that you’ve learned how to fetch live data using the OpenWeatherMap API, you’re ready to develop a simple weather application using it. Building a weather application can help you to apply what you know and hone your Python skills.
Getting your hands dirty on practical projects can make you a better developer. You can develop some other Python projects like a login system, quiz app, or URL shortener to solidify your Python development skills.
