What makes LINQ?

Building LINQ expressions with the Enumerable class

All generic collections, such as List<T>, Dictionary<TKey, TValue>, Stack<T>, and Queue<T>, implement IEnumerable<T>, so they can be queried and manipulated with LINQ too.

Enumerable defines more than 50 extension methods, as summarized in the following table:

Understanding deferred execution

1. Use your preferred code editor to create a new solution/workspace named Chapter11.
2. Add a console app project, as defined in the following list:
   1. Project template: Console Application / console
   2. Workspace/solution file and folder: Chapter11
   3. Project file and folder: LinqWithObjects
3. In Program.cs, delete the existing code and statically import Console.
4. Add statements to define a sequence of string values for people who work in an office, as shown in the following code:

   // a string array is a sequence that implements IEnumerable<string>
   string[] names = new[] { "Michael", "Pam", "Jim", "Dwight", 
     "Angela", "Kevin", "Toby", "Creed" };
   WriteLine("Deferred execution");
   // Question: Which names end with an M?
   // (written using a LINQ extension method)
   var query1 = names.Where(name => name.EndsWith("m"));
   // Question: Which names end with an M?
   // (written using LINQ query comprehension syntax)
   var query2 = from name in names where name.EndsWith("m") select name;
   

5. To ask the question and get the answer, i.e. execute the query, you must materialize it by either calling one of the "To" methods like ToArray or ToLookup or by enumerating the query, as shown in the following code:

   // Answer returned as an array of strings containing Pam and Jim
   string[] result1 = query1.ToArray();
   // Answer returned as a list of strings containing Pam and Jim
   List<string> result2 = query2.ToList();
   // Answer returned as we enumerate over the results
   foreach (string name in query1)
   {
     WriteLine(name); // outputs Pam
     names[2] = "Jimmy"; // change Jim to Jimmy
     // on the second iteration Jimmy does not end with an M
   }
   

6. Run the console app and note the result, as shown in the following output:

   Deferred execution
   Pam
   

Before we get too deep into the weeds, let's slow down and look at some common LINQ extension methods and how to use them, one at a time.

Filtering entities with Where

The Func<string, bool> delegate tells us that for each string variable passed to the method, the method must return a bool value. If the method returns true, it indicates that we should include the string in the results, and if the method returns false, it indicates that we should exclude it.

Targeting a named method

Let's define a method that only includes names that are longer than four characters:

Simplifying the code by removing the explicit delegate instantiation

1. To help you learn by seeing progressively improved code, copy and paste the query
2. Comment out the first example, as shown in the following code:

   // var query = names.Where(
   //   new Func<string, bool>(NameLongerThanFour));
   

3. Modify the copy to remove the explicit instantiation of the delegate, as shown in the following code:

   var query = names.Where(NameLongerThanFour);
   

4. Run the code and note that it has the same behavior.

Targeting a lambda expression

1. Copy and paste the query, comment the second example, and modify the query, as shown in the following code:

   var query = names.Where(name => name.Length > 4);
   

   Note that the syntax for a lambda expression includes all the important parts of the NameLongerThanFour method, but nothing more. A lambda expression only needs to define the following:

   • The names of input parameters: name
   • A return value expression: name.Length > 4

   The type of the name input parameter is inferred from the fact that the sequence contains string values, and the return type must be a bool value as defined by the delegate for Where to work, so the expression after the => symbol must return a bool value.

   The compiler does most of the work for us, so our code can be as concise as possible.

2. Run the code and note that it has the same behavior.

Sorting entities

Other commonly used extension methods are OrderBy and ThenBy, used for sorting a sequence.

Sorting by a single property using OrderBy

To put the longest name first, you would use OrderByDescending.

Sorting by a subsequent property using ThenBy

1. Add a call to the ThenBy method at the end of the existing query, as shown highlighted in the following code:

   var query = names
     .Where(name => name.Length > 4)
     .OrderBy(name => name.Length)
     .ThenBy(name => name);
   

2. Run the code and note the slight difference in the following sort order. Within a group of names of the same length, the names are sorted alphabetically by the full value of the string, so Creed comes before Kevin, and Angela comes before Dwight, as shown in the following output:

   Creed 
   Kevin 
   Angela 
   Dwight 
   Michael
   

Declaring a query using var or a specified type

Filtering by type

Imagine that you have a sequence of exceptions. There are hundreds of exception types that form a complex hierarchy, as partially shown in Figure 11.3:

Figure 11.3: A partial exception inheritance hierarchy

Let's explore filtering by type:

Working with sets and bags using LINQ

With Zip, if there are unequal numbers of items in the two sequences, then some items will not have a matching partner. Those without a partner, like Jared, will not be included in the result.

For the DistinctBy example, instead of removing duplicates by comparing the whole name, we define a lambda key selector to remove duplicates by comparing the first two characters, so Jared is removed because Jack already is a name that starts with Ja.

So far, we have used the LINQ to Objects provider to work with in-memory objects. Next, we will use the LINQ to Entities provider to work with entities stored in a database.

Using LINQ with EF Core

Building an EF Core model

Filtering and sorting sequences

SELECT * FROM Products;

In Chapter 10, Working with Data Using Entity Framework Core, you learned how to log the SQL commands executed against SQLite so that you could see this for yourself.

Projecting sequences into new types

public class Person
{
  public string Name { get; set; }
  public DateTime DateOfBirth { get; set; }
}
Person knownTypeObject = new()
{
  Name = "Boris Johnson",
  DateOfBirth = new(year: 1964, month: 6, day: 19)
};

C# 3.0 and later allow instances of anonymous types to be instantiated using the var keyword, as shown in the following code:

var anonymouslyTypedObject = new
{
  Name = "Boris Johnson",
  DateOfBirth = new DateTime(year: 1964, month: 6, day: 19)
};

Although we did not specify a type, the compiler can infer an anonymous type from the setting of two properties named Name and DateOfBirth. The compiler can infer the types of the two properties from the values assigned: a literal string and a new instance of a date/time value.

This capability is especially useful when writing LINQ queries to project an existing type into a new type without having to explicitly define the new type. Since the type is anonymous, this can only work with var-declared local variables.

Let's make the SQL command executed against the database table more efficient by adding a call to the Select method to project instances of the Product class into instances of a new anonymous type with only three properties:

Joining and grouping sequences

Joining sequences

Group-joining sequences

Aggregating sequences

Sweetening LINQ syntax with syntactic sugar

Consider the following array of string values:

string[] names = new[] { "Michael", "Pam", "Jim", "Dwight", 
  "Angela", "Kevin", "Toby", "Creed" };

To filter and sort the names, you could use extension methods and lambda expressions, as shown in the following code:

var query = names
  .Where(name => name.Length > 4)
  .OrderBy(name => name.Length)
  .ThenBy(name => name);

Or you could achieve the same results by using query comprehension syntax, as shown in the following code:

var query = from name in names
  where name.Length > 4
  orderby name.Length, name 
  select name;

The compiler changes the query comprehension syntax to the equivalent extension methods and lambda expressions for you.

The select keyword is always required for LINQ query comprehension syntax. The Select extension method is optional when using extension methods and lambda expressions because if you do not call Select, then the whole item is implicitly selected.

Not all extension methods have a C# keyword equivalent, for example, the Skip and Take extension methods, which are commonly used to implement paging for lots of data.

A query that skips and takes cannot be written using only the query comprehension syntax, so we could write the query using all extension methods, as shown in the following code:

var query = names
  .Where(name => name.Length > 4)
  .Skip(80)
  .Take(10);

var query = (from name in names
  where name.Length > 4
  select name)
  .Skip(80)
  .Take(10);

Using multiple threads with parallel LINQ

Creating an app that benefits from multiple threads

We will use operating system tools to monitor the CPU and CPU core usage. If you do not have multiple CPUs or at least multiple cores, then this exercise won't show much!

Using Windows

Using macOS

For all operating systems

You will learn more about managing multiple threads in Chapter 12, Improving Performance and Scalability Using Multitasking.

Creating your own LINQ extension methods

We will improve the Average extension method as an example. A well-educated school child will tell you that average can mean one of three things:

• Mean: Sum the numbers and divide by the count.
• Mode: The most common number.
• Median: The number in the middle of the numbers when ordered.

1. In the LinqWithEFCore project, add a new class file named MyLinqExtensions.cs.
2. Modify the class, as shown in the following code:

   namespace System.Linq; // extend Microsoft's namespace
   public static class MyLinqExtensions
   {
     // this is a chainable LINQ extension method
     public static IEnumerable<T> ProcessSequence<T>(
       this IEnumerable<T> sequence)
     {
       // you could do some processing here
       return sequence;
     }
     public static IQueryable<T> ProcessSequence<T>(
       this IQueryable<T> sequence)
     {
       // you could do some processing here
       return sequence;
     }
     // these are scalar LINQ extension methods
     public static int? Median(
       this IEnumerable<int?> sequence)
     {
       var ordered = sequence.OrderBy(item => item);
       int middlePosition = ordered.Count() / 2;
       return ordered.ElementAt(middlePosition);
     }
     public static int? Median<T>(
       this IEnumerable<T> sequence, Func<T, int?> selector)
     {
       return sequence.Select(selector).Median();
     }
     public static decimal? Median(
       this IEnumerable<decimal?> sequence)
     {
       var ordered = sequence.OrderBy(item => item);
       int middlePosition = ordered.Count() / 2;
       return ordered.ElementAt(middlePosition);
     }
     public static decimal? Median<T>(
       this IEnumerable<T> sequence, Func<T, decimal?> selector)
     {
       return sequence.Select(selector).Median();
     }
     public static int? Mode(
       this IEnumerable<int?> sequence)
     {
       var grouped = sequence.GroupBy(item => item);
       var orderedGroups = grouped.OrderByDescending(
         group => group.Count());
       return orderedGroups.FirstOrDefault()?.Key;
     }
     public static int? Mode<T>(
       this IEnumerable<T> sequence, Func<T, int?> selector)
     {
       return sequence.Select(selector)?.Mode();
     }
     public static decimal? Mode(
       this IEnumerable<decimal?> sequence)
     {
       var grouped = sequence.GroupBy(item => item);
       var orderedGroups = grouped.OrderByDescending(
         group => group.Count());
       return orderedGroups.FirstOrDefault()?.Key;
     }
     public static decimal? Mode<T>(
       this IEnumerable<T> sequence, Func<T, decimal?> selector)
     {
       return sequence.Select(selector).Mode();
     }
   }
   

Trying the chainable extension method

1. In Program.cs, in the FilterAndSort method, modify the LINQ query for Products to call your custom chainable extension method, as shown highlighted in the following code:

   DbSet<Product>? allProducts = db.Products;
   if (allProducts is null)
   {
     WriteLine("No products found.");
     return;
   }
   IQueryable<Product> processedProducts = allProducts.ProcessSequence();
   IQueryable<Product> filteredProducts = processedProducts
     .Where(product => product.UnitPrice < 10M);
   

2. In Program.cs, uncomment the FilterAndSort method and comment out any calls to other methods.
3. Run the code and note that you see the same output as before because your method doesn't modify the sequence. But you now know how to extend a LINQ expression with your own functionality.

Trying the mode and median methods

There are four products with a unit price of $18.00. There are five products with 0 units in stock.

Working with LINQ to XML

Generating XML using LINQ to XML

Reading XML using LINQ to XML

You might want to use LINQ to XML to easily query or process XML files:

Practicing and exploring

Test your knowledge and understanding by answering some questions, get some hands-on practice, and explore with deeper research into the topics covered in this chapter.

Exercise 11.1 – Test your knowledge

Answer the following questions:

1. What are the two required parts of LINQ?
2. Which LINQ extension method would you use to return a subset of properties from a type?
3. Which LINQ extension method would you use to filter a sequence?
4. List five LINQ extension methods that perform aggregation.
5. What is the difference between the Select and SelectMany extension methods?
6. What is the difference between IEnumerable<T> and IQueryable<T>? And how do you switch between them?
7. What does the last type parameter T in generic Func delegates like Func<T1, T2, T> represent?
8. What is the benefit of a LINQ extension method that ends with OrDefault?
9. Why is query comprehension syntax optional?
10. How can you create your own LINQ extension methods?

Exercise 11.2 – Practice querying with LINQ

In the Chapter11 solution/workspace, create a console application, named Exercise02, that prompts the user for a city and then lists the company names for Northwind customers in that city, as shown in the following output:

Enter the name of a city: London 
There are 6 customers in London: 
Around the Horn
B's Beverages 
Consolidated Holdings 
Eastern Connection 
North/South
Seven Seas Imports

Then, enhance the application by displaying a list of all unique cities that customers already reside in as a prompt to the user before they enter their preferred city, as shown in the following output:

Aachen, Albuquerque, Anchorage, Århus, Barcelona, Barquisimeto, Bergamo, Berlin, Bern, Boise, Bräcke, Brandenburg, Bruxelles, Buenos Aires, Butte, Campinas, Caracas, Charleroi, Cork, Cowes, Cunewalde, Elgin, Eugene, Frankfurt a.M., Genève, Graz, Helsinki, I. de Margarita, Kirkland, Kobenhavn, Köln, Lander, Leipzig, Lille, Lisboa, London, Luleå, Lyon, Madrid, Mannheim, Marseille, México D.F., Montréal, München, Münster, Nantes, Oulu, Paris, Portland, Reggio Emilia, Reims, Resende, Rio de Janeiro, Salzburg, San Cristóbal, San Francisco, Sao Paulo, Seattle, Sevilla, Stavern, Strasbourg, Stuttgart, Torino, Toulouse, Tsawassen, Vancouver, Versailles, Walla Walla, Warszawa

Exercise 11.3 – Explore topics

Use the links on the following page to learn more details about the topics covered in this chapter: