<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:12.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:12.0pt;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt"># ASPLOS 2021<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">For my meetup requirement, I attended the 2021 ASPLOS Conference which is focused on architectural support for programming languages and operating systems. The primary reason for me taking this course is to
familiarize myself in the management of systems and use it to better understand systems architecting. For this reason exactly the conference, and more specifically the section on microservices, stuck out to me. I started by attending a paper presentation on
microservices from the University of Texas at Austin by Zhipeng Jia and Emmett Witchel. The paper is available at https://dl.acm.org/doi/abs/10.1145/3445814.3446701 .<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">## Nightcore: Efficient and Scalable Serverless Computing for Latency-Sensitive, Interactive Microservices<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">### Servless Microservices<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Microservices have gained increasing popularity in software architectures for building flexible, large-scale, online services. Specifically serverless funtions (FaaS), make it extremely easy to create statelss
functions in the cloud which can be RPC handlers compared to more complicated containerized RPC servers. The benefit of these is they have high elasticity and good pricing as well as contribute to a composable software design. However, the primary downside
of these serverless functions is they have a millisecond start up delay on runtime making them slow down high performing services. Examples of these are AWS Lambda which are 11.5x slower than RPC servers at media latency.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">### Performance Goals for Nightcore<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Function Execution time must be in 100s of microseconds, invocation latency must be under 100 microseconds, and should support ~100K/s for invocation rate.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">### Vulnerable to Killer Microseconds<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">The software is vulnerable to a handful of microsecond-scale events in networking, TCP/IP stack, RPC protocol, context switch, thread scheduling, and more outlined in `Attack of the Killer Microseconds` from
the March 2017 Communications of the ACM proceeding.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">### Nightcore Design<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">#### Techniques<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">* Optimize locality of internal function calls<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">* High optimizations for local I/Os<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> * Low-latency message channels<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> * Event-driven concurrency<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">* Managing per-microservice concurrency to mitigate load variation<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">#### FaaS Runtime<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">A request is made the Frontend to load balanced backends as adopted by Apache OpenWhisk and OpenFaaS. This architecture is limited in that it communicates back to the frontend but Nightcore optimizes locality
for internal function call by chaining functions with internal function calls in the backend infrastructure.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">#### Nightcore Overview<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">API gateway to per-function dispatching queues on a worker server which then direct invocations to Nightcore's runtime library which does fast paths for internal function calls. This can be done in a VM or
Docker container. The speaker provides an example run through of an internal function request. They created IPC primitive for function worker I/Os that creates a low latency message channel which is 4x quicker than standard gRPC.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">#### Benchmarks / Performance<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Compared to RPC servers and OpenFaaS with C++, Go, Node.js, and Python using EC2 instances. They use trial applications including a Social Network, Movie Reviewing Platform, Hotel Reservation, and Hipster
Shop. They use these example services to measure throughput and Nightcore scores best on all their measurements.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">## Conclusion / Takeaways<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">This is important to system administrators as they wear many hats and should understand the system architectures they are working with. If latency is a large problem in your microservice architecture, you
can try to optimize using the techniques they showed or just by using their runtime. Also, I attend a handful of other cool events including in depth papers on programming languages and a Q&A with Kathyn McKinley (a Principal Research Scientist at Google)
where she talked about cloud, parallel computing, and so much more. Overall, i'd recommend a conference like this to anyone in the course which was a steal for only $15.<o:p></o:p></span></p>
</div>
</body>
</html>