Grpc solution structure with multiple projects and references to proto files // Nikolay Krondev blog

Intro

Grpc is getting more popular. Microsoft is promoting it with each new release of .net. But there is one thing that I personally really don’t like with all the manuals on Internet (even Microsoft documentation) and it is that you have to manually copy your proto files between server and client. There is a .net cli tool that helps you add it either as a reference to a file or to a url somewhere on the internet. But again… it is 2021, do we really need to do it this way, Microsoft?. In this post I will show you how to make your life easier with complex project structure (when you have several protos referencing eachother) and extracting grpc service functionality into abstraction above that can be delivered via nuget.

Note: Working solution is available at my github repo BlogpostDemo-Grpc-project-structure

Structure

What I basically will do is:

extract all proto files into a folder with a proper structure
create a nuget library for client and server (for even better separation you can have two nuget libraries with different logic)
reference extracted library in both client and server

Shared protos

Instead of following Microsoft’s tutorial I can first create shared proto files that can be used in my dll library that I can later on use as nuget packages.

1
2
3
4
5
6
7
8
9


> mkdir shared_protos
> mkdir shared_protos/Greeter
> mkdir shared_protos/Greeter/Enums
> touch shared_protos/Greeter/Enums/Status.proto
> mkdir shared_protos/Greeter/Messages
> touch shared_protos/Greeter/Messages/HelloRequest.proto
> touch shared_protos/Greeter/Messages/HelloReply.proto
> mkdir shared_protos/Greeter/Services
> touch shared_protos/Greeter/Services/GreeterService.proto

Status.proto file content:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


syntax = "proto3";

package krondev.protobuf.Greeter;
option csharp_namespace = "Krondev.Greeter";

enum Status {
  Status_NotSet = 0;
  Success = 1;
  Fail = 2;
}

HelloRequest.proto file content:

1
2
3
4
5
6
7
8


syntax = "proto3";

package krondev.protobuf.Greeter;
option csharp_namespace = "Krondev.Greeter";

message HelloRequest {
  string name = 1;
}

HelloReply.proto file content:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


syntax = "proto3";

import "Greeter/Enums/Status.proto";

package krondev.protobuf.Greeter;
option csharp_namespace = "Krondev.Greeter";

message HelloReply {
  string message = 1;
  krondev.protobuf.Greeter.Status Status = 2;
}

GreeterService.proto file content:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


syntax = "proto3";

import "Greeter/Messages/HelloRequest.proto";
import "Greeter/Messages/HelloReply.proto";

package krondev.protobuf.Greeter;
option csharp_namespace = "Krondev.Greeter";

service GreeterService {
  rpc SayHello (HelloRequest) returns (HelloReply);
}

Nuget

Then I will create a nuget library:

1

> dotnet new classlib -o Krondev.Greeter

Now to the most interesting part, I need to update csproj file to reference this proto file and update by using Protobuf ItemGroup:

1
2
3
4
5


<ItemGroup>
    <Protobuf Include="../shared_protos/Greeter/Enums/*.proto" Link="Protos/Enums/*.proto" ProtoRoot="../shared_protos" />
    <Protobuf Include="../shared_protos/Greeter/Messages/*.proto" Link="Protos/Messages/*.proto" ProtoRoot="../shared_protos" />
    <Protobuf Include="../shared_protos/Greeter/Services/*.proto" Link="Protos/Services/*.proto" ProtoRoot="../shared_protos" />
</ItemGroup>

As you can see I don’t reference files one by one, instead I use asterisk mask so all files from the specified folder will be referenced.

Alternative solution

Another solution could possibly have been to include all proto files as a file reference in the nuget packages directly, but this however is not approved by the Nuget team. So we will just have to wait until they have resolved all the security issues. You can follow the discussion on this issue on GitHub https://github.com/grpc/grpc-dotnet/issues/183

And of course I need Grpc libraries to be able to generate c# code:

1
2
3
4
5
6


<ItemGroup>
  <PackageReference Include="Google.Protobuf" Version="3.17.3" />
  <PackageReference Include="Grpc.Core.Api" Version="2.34.0" />
  <PackageReference Include="Grpc.Net.Client" Version="2.34.0" />
  <PackageReference Include="Grpc.Tools" Version="2.39.1" PrivateAssets="All" />
</ItemGroup>

Now I had to build the project so compiler could generate all the grpc c# code for me, so that I can use it later in GreeterClient.cs:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78


using System;
using System.Threading;
using System.Threading.Tasks;
using Grpc.Core;
using Grpc.Net.Client;
using Krondev.Greeter.Configuration;

namespace Krondev.Greeter.Clients
{
    public class GreeterClient : 
        GreeterService.GreeterServiceBase,
        IGreeterClient,
        IAsyncDisposable
    {
        private readonly GreeterService.GreeterServiceClient _baseClient;
        
        private readonly string _fullName;
        private readonly GrpcChannel _channel;
        
        public GreeterClient(ChannelCredentials channelCredentials,
            GreeterClientOptions options = default)
        {
            if (channelCredentials == null)
                throw new ArgumentNullException(nameof(channelCredentials));

            options ??= new GreeterClientOptions();
            
            _channel = GrpcChannel.ForAddress(options.Host + ':' + options.Port, new GrpcChannelOptions
            {
                Credentials = channelCredentials,
                HttpHandler = options.HttpHandler
            });
            _baseClient = new GreeterService.GreeterServiceClient(_channel);

            _fullName = GetType().FullName;
        }
        
        public async Task<HelloReply> SayHello(HelloRequest request, CancellationToken cancellationToken = default)
        {
            // do your extra pre-logic...
            try
            {
                using var call = _baseClient.SayHelloAsync(
                    request,
                    CreateTraceMetaData(),
                    cancellationToken: cancellationToken);
                var response = await call.ResponseAsync;
                return response;
            }
            catch (Exception)
            {
                // do your extra logging here 
                throw;
            }
        }

        private static Metadata CreateTraceMetaData()
        {
            var metadata = new Metadata
            {
                {
                    "correlation-id",
                    Guid.NewGuid().ToString()   // replace with your logic
                }
            };

            return metadata;
        }
        
        public async ValueTask DisposeAsync()
        {
            if (_channel != null)
            {
                await _channel.ShutdownAsync();
            }
        }
    }
}

Notice that I now have control over the execution, which means I can also implement some pre-checks in the // do your extra pre-logic... comment and if you want to - handle exceptions.

Server

Typical server code can be generated by dotnet cli:

1

> dotnet new grpc -o server

I just need to remove the Protos/greet.proto file, reference my new nuget library and inherit GreeterService.cs class from my base class:

1
2


public class GreeterService : 
         Krondev.Greeter.GreeterService.GreeterServiceBase

Note: if you are working in mac environment you need to add some additional code, to do so, follow Microsoft recommendations

Client

Client can be whatever we want. In this example I will create a simple console application (the same as Microsoft documentation does). If you use asp.net core - do not forget to register all the necessary services in ConfigureServices method in Startup.cs and do not instantiate client directly as it is shown in the console sample.

Create a new console app:

1

> dotnet new console -o client

And replace original content of Program.cs completely with:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39


using System;
 using System.Net.Http;
 using System.Runtime.InteropServices;
 using System.Threading.Tasks;
 using Grpc.Core;
 using Krondev.Greeter;
 using Krondev.Greeter.Clients;
 using Krondev.Greeter.Configuration;

 namespace client
 {
     class Program
     {
         private static async Task Main()
         {
             if (RuntimeInformation.IsOSPlatform(OSPlatform.OSX)) {
                 // The following statement allows you to call insecure services. To be used only in development environments.
                 AppContext.SetSwitch("System.Net.Http.SocketsHttpHandler.Http2UnencryptedSupport", true);
             }

             var httpHandler = new HttpClientHandler();
             // Return `true` to allow certificates that are untrusted/invalid
             httpHandler.ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator;

             await using var client = new GreeterClient(ChannelCredentials.Insecure, new GreeterClientOptions
             {
                 Host = "http://localhost",
                 Port = 5000,
                 HttpHandler = httpHandler
             });

             var reply = await client.SayHello(
                 new HelloRequest { Name = "GreeterClient" });
             Console.WriteLine("Greeting: " + reply.Message + " with status " + reply.Status);
             Console.WriteLine("Press any key to exit...");
             Console.ReadKey();
         }
     }
 }

If we skip some MacOS specific code what I basically do here is just instantiate a new instance of my nuget client with options. That’s it, pretty clean and simple.

Summary

In this post I have shown a better grpc project(s) structure where instead of direct proto files references we can have a nuget package that can easily be used in several clients and servers.

Since protobuf already provides backwards compatibility we can have different versions of our nuget package and all old services will not get broken.

Please check my repository for the fully working code https://github.com/nikolaykrondev/BlogpostDemo-Grpc-project-structure