Google Cloud Platform Live: DevOps at Google Speed & Tools for You


[MUSIC PLAYING] RAE WANG: Good
afternoon, everyone. Welcome back. It’s always hard to be the
session right after lunch. How was food? Was it good? Right. Google is known
for our good food. But did you eat too much so
that now you’re a little sleepy? I hope not. Well actually, I’m not worried. Because what we’re
going to talk about is going to be real exciting and
close and dear to your heart. Dev Ops. So we’re going to
be able to have you engaged for the next hour. We’re going to talk about how
to not only write code, but also shepherd it to production,
maintain your live site, and troubleshoot and
debug successfully. I’m Rae Wang. I’m a product manager on
the Google Cloud developer experience team. And I have here
with me Chris Smith, who is a tech lead on the
team and my demo buddy today. The clicker. OK. I used to work on the back
end of a large scale service. It was real exciting,
the kind of problems you get only when
a scale is large, and the number of customers
you get to impact. But it was also frustrating. We had a lot of
really good ideas, we wanted to launch a new app
and add features and impact customers. But in the end, the time that
we got to spend writing code was about less
than half the time. So what were we doing, right? We were caught in new releases,
building tests and trying to deploy to production,
debug, the live site duties. Not to mention
when I was on call, I was paged around the clock. That wasn’t fun, I
didn’t have a life. Here at Google, we believe
that a Cloud is not just about giving you raw VM
and storage resources. You deserve to have
a platform that can take care of all the time
consuming, repetitive Dev Ops tasks so that you
can be freed up to work on what’s
more important. To launch that new app, to
build a new business idea, and still have a life, right? And we want to share
this dream with you. We want to invite you to join us
as we begin this journey to set a higher bar for what
Cloud can do for you. In the next hour,
you’re going to be able to see some of the first
steps that we have made, as well as the directions
that we’re heading. And now first, I
have a treat for you. So some of you might know that
Google runs some pretty large apps, like google.com,
Gmail, YouTube, do they ring any bells? You use them, right? Our developers work
efficiently to build and run these applications
because they have access to a great stack of tools
that Google spent the last 15 years building and perfecting. Now today, we have the
privilege of hearing from the person at
the heart of this. Melody Meckfessel, our
engineering director for developer infrastructure. MELODY MECKFESSEL:
Thanks a lot, Rae. So hi, I’m Melody. I lead a team at Google
focused on developer tooling and infrastructure. So what we do is we organize
software development at Google. And hopefully make it fun. But most importantly,
make it fast and scalable. And it’s all about
Google engineers. So from the time that engineers
sit down to write code, to search it, to
version it, and then to build, test, and release
it reliably to our Cloud. Developer tooling
enables engineers to do all of those things. So our whole mission is to
make engineers lives easier so engineers can focus on code. And as you heard Urs
say in the keynote, we want you to focus on
what you love, writing code. But it’s not just
about Google engineers. It’s about all of you. We want to take what we’ve built
internally and offer it to you. So I’m going to talk a
little bit now about some of our internal tools to
set the stage for the things that we want to offer you. OK. So how does Google build? Well, our build system supports
a variety of languages. And we have a recipe for
how to build our software. We enable engineers to
express relationships between libraries and binaries. And to get fast
feedback on their build. Well you might be asking,
how does this really happen? Well under the hood, just
think massive parallelism. Build actions being
farmed out to thousands of dedicated worker
machines getting the full power of
Google’s Cloud. But also, we cache results
across machines and users so that there’s a
transparent speed up for everyone when
somebody builds. So let me tell you a story. So before I joined Google
it was not uncommon for me to, say, kick off a
build before I left the office. I’d go home and I’d get
a good night’s sleep and I’d come back, and
then I’d see what happened. Were their breakages? I’d have to go figure that out. And then I’d have to
start it all over again. So when I joined Google I
thought wow, this is amazing. This is not where I came
from in terms of velocity, and speed, and feedback
for me as an engineer. And I’ve seen over the years,
the speed and the scale increase at Google. So now talk to nooglers,
so new Google engineers. They come in and they
say the same thing to me. Melody, wow. These developer
tools are amazing. But wait, one month
later, it’s too slow. It’s too slow to get a build
result in under a minute or in a matter of
seconds, right? It’s not enough. So that point I
want to make there is that the speed and the scale
are something that we do well. And we’re continuing
to set the bar higher and higher for ourselves. OK. What does Google do for testing? Because testing and developing
safely go hand in hand. Right? So we test the entire code
base after every repo update. And the other thing
is that we provide testing before submit so that
we can identify breakages before they even happen. I think that’s pretty cool. Before submit, you know
what’s going to be broken. The other thing that
we do is we know that we need to test in a
variety of environments. So whether it’s development,
or staging, or testing, we make it easy
for engineers to be able to switch environments
and do the testing that they need to do. We also need to think about
things like automated web app testing, right? All the various OS and
browser combinations that our users use, we need to
be able to test all of those. And so we run millions
of tests a day and 20 plus OS or
browser combinations to make sure what’s going out
the door is the best quality. OK. So let’s talk a little bit about
what Google does internally to manage our code. So we have a single code
tree with very fast access. So I know many of you–
DVCS, raise your hands? There’s audience participation. I should’ve warned you. I’m going to be asking
you lots of questions. OK. Just one? Really? OK. All right. So even though we
have this single tree, we do want to provide
a lot of flexibility around workflows for engineers. So we support– some get
workflows internally. We merge early and often. We’re constantly doing
integrations with the code. The other thing that I
think is pretty amazing is that the code
base is very open. Engineers can go look, view,
modify, experiment with code across the entire repo. So I think that’s
pretty amazing, right? That level of openness. As engineers, I think I can
safely say, we love languages. Right? We love languages. So we support a
variety of languages. C++, Go, Java, Python, right? It’s important. We need to pick the appropriate
language for the problem that we’re trying to solve. OK. Well, what does this actually
mean for an engineer? Having a single code tree
means that we can do things like globally analyze
and refactor our code. And with over 300 million
plus lines of code at Google, keeping the code tree clean
is very important to us. So here’s another story. I’ve seen this multiple times in
our history of changing an API. What do we need to do? We need to find all
the usage of that API. We generate CLs to send
to the code owners. Those need to go through review,
and testing, and push out. These large scale code changes
and massive refactorings have enabled us to
increase our velocity. And we get that benefit
with a single code tree. Fast access means
that engineers can answer all kinds of interesting
questions about the code. Who’s using my API? What’s this library? Look for another piece of code. And merging early
and often enables us to do easy rollback
when things go wrong. And we know we’re
going to break things, so we need to make sure
we can roll back safely. Having the code be
open means that we can help each other out. We can help fix bugs. An issue comes up, we can
bring together the right people to get it fixed. And quality is very
important to us. So we do have
mandatory code review by experts and our
code developers. So I’m going to take a pause
there for a second and ask you, how many builds do you
think Google does a day? Audience participation. How many? Millions. OK. How many tests? One build. That’s a good one. How many tests? Anyone? Bueller? OK. OK. All right, well here’s
some of the stats that we’re talking about. So around 800,000
builds a day leading to 2 petabytes of build output. 25,000 changelists in a day
with– per minute ranging from 20 plus per minute to
60 plus per minute at peak. So the tooling that
we’ve built at Google, really, the whole focus
is to enable velocity. And speed and scale
are critical for us. So working at speed and
scale, it’s a challenge. And the engineers keep– we keep
raising the bar for ourselves. But what’s it like? What’s the culture like? And I just wanted to point
out a couple of things. This idea of having
the code base be open, collaboration and
co-development are key. With this accountability to
help each other fix issues. Sharing code across
products means that engineers don’t have to
write code that already exists. We have common libraries. And common services. And different pieces
that we can reuse, and that leads to velocity. And engineers have autonomy. Engineers can make
their own choices around what makes
them productive. So how many Emacs users? Audience participation. OK All right. How about Eclipse? All right! Dare I say Vi? Yeah! There we go! So we know that we need to offer
a variety of tools and options so that engineers
have this choice to pick what works for them. And near and dear to my
heart, as an engineering team building tools
and infrastructure for other engineers,
we need to make sure it makes us productive. We use it ourselves. We do this thing
called dog fooding where we make sure that it
works before sending it out to our users. So I don’t know if
you can tell, I’m a little proud of what
we’ve built internally. I’m proud of it. And it sounds
pretty good, right? Well, we struggle. And not to air dirty
laundry here, but I will. It’s never fast enough. Every single engineer that I
talk to, it’s not fast enough. The build is too slow. Engineers want and expect
almost instantaneous results for what’s broken. What test is broken? So that you fix it so we can
get features out to our users. The rate of change
is so fast, it’s hard to keep up some times. So if you have
dependencies in storage or in other areas
of the stack, it’s frustrating when things break. It can slow us down. And that’s something
that we’re working on. Quality processes are
very important us. They help us a lot. Sometimes they slows down. It takes too long to
turn things around. And we want to keep this focus
on automation and velocity for engineers. And finally, I just
want to highlight that finding and fixing
bugs– it’s tough for us too. I need to go look at this
log file and this log file. And I got an alert. I’m not sure that’s
related, and I need to figure out what
needs to be monitored. So there’s a session
right after this to talk about the
insight we want to give you into your service. But I just want to acknowledge
that this is something that we struggle with. And we want to make it
better for everyone. So we think you deserve better. As you heard Urs
say today, we’re not keeping any of these
tools to ourselves. So whether you’re
Gmail or Snapchat, we’re going to focus on making
the best platform that everyone can build on. And I’m really excited
about the opportunity to take the experience,
knowledge, and tools that we have internally and
offer these to you. We want to develop a
community for third parties that’s easy for third
parties and developers to plug in functionality,
to innovate in new markets and tools, and to raise the
overall value of the ecosystem. To make it better for all of us. So with that, I’m going to
turn it over to Rae and Chris to talk about the specifics
of what we’re going to launch. Productive developer tooling,
continuous delivery system, and better Dev Ops. Thank you so much
for being here today. RAE WANG: So as we
just heard from Melody that Dev Ops is
hard for everyone. It’s hard at Google too. Regardless of whether it’s
making 800,000 builds at Google or it’s your own company
making three builds a day. It’s hard. It’s time consuming. Developers all need support
from great platforms to be productive and innovative. So that’s why we
have started now building these Dev Ops tools,
not only for Google developers, but more importantly, for you,
the Google Cloud customers. Let’s dive into them. First, I want to
talk about code. The soul of your application
lies in the code you write. So you need a great
experience working on your code on a cloud. We did a lot of
user research study. That’s when we go out and
talk to many developers and actually sit there for
hours and watch how you work. And we learned there are a few
things that your favorite IDEs don’t do for you yet. First of all, how
many of you only work from a single
desktop at a time? One? OK. No laptops, no
phone, no tablets? So these days, a
lot of applications need to be up 24/7. But we can’t be chained
to our desk for that long. You have a life. You’re on the go. Luckily, you have devices. And your code needs to be
accessible from where you are. The other thing we observed
was that many of you need to switch between multiple
windows to get your work done. Because it’s not just your code. You also want to look at
logs and traces and metrics and analytics. I know the two big
monitors with eight screens that gets your eyes
busy all the time. That probably looks
cool, and it’s probably how Hollywood wants
to think about it, but you know and I know that’s
not good for your productivity. The context switching
slows you down. So the ideal Cloud
should offer you easy access to that entire
stack of data put in a context that you want. We don’t like it
when things go wrong. We really don’t like it. But it’s life, and it happens. Today when stuff goes wrong,
it’s often really difficult to find the culprit
code because– just think about how you do that. First, you have to figure
out, which releases have been deployed
to which environment. At what time? By who? And then you have
chase down which code commits went
into these releases. We think a powerful
diffing tool should easily help you answer the
question of what has changed without requiring
you to manually look into these different
sources of data. So now let’s see the real thing. Chris is going to show some
demos of the features– we’ll be able to solve these problems CHRIS SMITH: Hello. I’m Chris Smith. I’m a tech lead on
the team here and I’m very excited about
developer tools, and tooling, and
being more productive. But I’m going to let
you on a secret today, and that’s my first love
is game development. Which is why I am super stoked
today to launch my brand new, super exciting,
original IP, Fluttery Bird. Now of course– right? Yeah! Now this may look like a
cheap knockoff of Flappy Bird, and it may seem like a cheap
knockoff of Flappy Bird, but I assure you it’s super
awesome and a lot of fun. In honesty, I can’t
take credit for it. This is a clone of
Clumsy Bird, which is also a clone of Flappy Bird. It’s clones all the way down. But what I have done
is added a backend using Google App Engine to keep
track of a worldwide leader board. I just of two points. And now if I go to
the high scores page, I can see those two points. Now I see a few of you may
be in the audience trying to play along. And I don’t see these other
scores, even if I refresh, because there’s a bug here. You see, earlier today
I got a text message from a friend that’s like, hey,
my scores aren’t showing up. What’s the deal? And we can verify this
very quick by just going in an incognito
window, playing real quick, scoring a point,
and then verifying that, alas, the score
isn’t being uploaded. So there’s a problem here
with my production app, and on the launch
day of Fluttery Bird. So what am I going to do? I’m going to fix this by
using all of the Dev Ops tools that we’ve built inside of
the Google Developers Console. So I’m going to
quickly load it up. Jump to my project. And the first thing
I’d like to show you is the Release History page. Open up Cloud Development,
and Release History. And this is showing
me all the deployments of my application, Flappy Bird. Something I’d like to
bring your attention to is the Changes column. So here’s a deployment where
I fix the high score sorting. If I open up the diffs
page, it will show me all the commits that were
included with this deployment. That is all the
changes that were made. As well as showing
me the diffs inline so I can quickly scan what’s
happened to my application. So using this
feature, if I go back, I could probably get a sense
on what the problem was from this dubious refactor
user service change. But I don’t want to give
away all my secrets. So another way to debug
or troubleshoot this app is to just go
straight to the logs. Loading up the new App
Engine Logs Viewer, I can filter this by error
and scroll all the way down. And sure enough, I see some
errors uploading high scores. Error uploading high
score, non-admin user attempting to
upload a high score. Well, that doesn’t sound right. I mean, sure as an admin
I can upload high scores, but I imagine that
other people would want to use this high
score leader board as well. OK. So what I’m about to show you
may be the greatest developer feature ever in the
history of mankind. Quick question. Right? Quick question. Why is this that
weird shade of blue? In fact, if I hover my mouse
over highscoreservice.java, that gets underlined
for some reason. It’s exactly what
you think it is. It’s a link. A hyperlink– boom–
from the runtime error logs directly to
your source code. Not only just some random
file, but the specific commit and the specific line
where that error occurred. So going straight from
the App Engine logs, I can jump right to the problem,
where it occurred to my code. And sure enough, some
bonehead developer on the team, this guy,
introduced a problem. So what I’m going to do
now is drive to the office, go to my desktop,
make the change, commit it, et cetera, et cetera. Or, right here on stage from
my web browser, or maybe my tablet, or possibly even a
phone, I can just click Edit. I’m going to make this change. Just comment out
this piece of code. And then I’m going to commit
it, all from my browser. All right. And this is going to commit. But not only that. It’s then going to
build my application, run its unit test,
and once everything’s green, deploy Fluttery Bird
worldwide, fixing the problem, and saving the day. I’ll talk more about our
release and deployment tools in a minute, but for now, I’d
like to give it back to Rae. RAE WANG: Now that
[INAUDIBLE] done, I just wanted to say
again, go Fluttery Bird. So now let’s look at a
recap of the features that you just say in the demo. And by the way, all the
features you just saw will be coming to you
in the next few weeks. To help you to easily work
with your code in a Cloud, we’re offering you
Google hosted Git repos. And you also saw
the browser based file viewer and code editor. And not only that, the ability
to checking your code right from there where your code is. To help bring together your
code and a runtime data, we’re offering you
log to source linking. That’s where you can go directly
from the message in your log file into the code
that caused the error. And that’s just the first step. We know that logs are not
the only thing you look at. There are also monitoring
metrics, other [INAUDIBLE] datas. And we’re thinking about
bringing all of them here so that your code can
become the canvas on which you can display and analyze your
application’s performance data. To help you answer the
question of what has changed, we’re giving you
not only code diff. That’s normal. Everybody does that. But more importantly,
deployment diff. So you can easily
tell that [INAUDIBLE] has happened between
two deployments. So now you have your code. What do you do when you have
your code after code is done? You don’t just keep it sitting
around your repo, right? You want to deploy it. That’s when it becomes real. So let’s talk about deployments. Deployment is the life
blood to your application. Deployment time is just
about the most exciting time because that’s when all
your hard work pays off. Right? That’s when your
app hits the market. Customers start using it. That’s when you become a hero. But it’s also a very scary time. That just tends to be
when things go wrong. How many of you have
had outages right when you deploy the
new version of an app? Happens, right? You’re with me there. So you definitely want to your
deployments to be reliable. That’s the most important thing. You want to have the confidence
that whatever you’re deploying to your production does not
have a critical bug in it. And you certainly don’t want to
be the one paged at 2 o’clock in the morning because you
just brought down the app. You want the
deployments to be fast. You don’t want to
wait for hours or days for your work to hit live
site to be used by customers. You want that happening
in a matter of minutes. The other thing that
fast deployments do for you is if you can
deploy and release fast, then you can end up doing many
frequent, smaller releases. Now why is that good for you? Let’s just imagine, if
you’re sending a release that contains 100 code
changes into production. Something goes wrong. What do you do? You have to roll back
that entire package, and then take it apart,
out of 100 changes, find the one that’s the
issue, cherry pick it out, put the 99 back together,
send it to production, and hope you got the right one. But if you can do fast and more
frequent and smaller releases, if your release contains
only one or two code changes, thing’s go wrong, roll
back, and you’re done. Right? And we don’t like
slow deployments. We want them to be fast. But there’s one
thing that can be even worse than
slow deployments. That’s not knowing
where your changes are. Have you had that happen to you? We all know what that’s like. Right? Your changes need to be
visible and predictable. When I used to run deployment
for a pretty big backend, I found my path of my job
became answering exactly three questions. Where are my changes? When they’re going
to get to production. And what has been deployed
to this environment. So after a while, I learned to
make myself more productive. I set up an auto
response message with some of the
answers in that. That made the line outside of
my office a little shorter. It shouldn’t be that hard. You shouldn’t need
a crystal ball just to tell when your changes
will get to production. And we certainly
don’t want you to keep pestering your release managers. And last but not least, we know
there are great best practices, having proven that you can
use tools to adapt to them. But at the end of the day,
every person and every team is different. You might want to build
and test differently than all the other developers. And you want to decide what
environments want to use. You might have specific
approval policies that you need to put into place. And we want to
support you there. We want to meet
you where you are. Now it’s an exciting
demo time again. CHRIS SMITH: All right. So we use this Push
To Deploy feature to roll out the fix to fluttery
bird, therefore saving the day. I see somebody who’s tried
to play it on their laptop, and evidently that
hasn’t fixed the issue. And we’ll see what the deal is. But first, let me
explain a little bit of how this is all happening. You see, last summer we
initially launched Push To Deploy. That is, a Git source
code repo that, whenever you make a commit
to, it automatically deploys it to App Engine. This is an alternative to
using the App Engine SDK and deploying manually. And it’s a way to be
more productive if you’re using Python or PHP, languages
that don’t require a build. But if you want to
use Java, or if you want to run some unit
tests before you deploy your application, well, then
you would be out of luck. So that’s why this
quarter we’ve launched Push To Deploy Plus Build. If I go to the Releases tab,
and under Configuration, you’ll be able to see the
different kinds of releases. Not only is there
Vanilla Push to Deploy, or Push to Deploy
Classic, but also offering a build machine
for Java Maven based builds, Java using the Gradle
build system, or Python with nose test, PHP
with PHP unit test. And I want to spend a
couple of minutes explaining part of the technology
on how this works. So whenever you make a commit
to a Push To Deploy Enabled repo, if it requires
build or test, will provision a Google
Compute Engine virtual machine for your source code to
do the build on the test. Normally this happens on demand,
but I’ve modified this project so that it’s still running
so I can show it to you. If in the developer console,
I quickly refresh this page. And it will show
a Compute Engine VM that is running the
actual build machine. In fact, if you look
over the past day, each spike in CPU utilization
is when I’ve triggered a build. And so this is the
actual machine. And you may be wondering,
so what’s actually doing the build? Is it a shell script? Is it some Google specialize
build farm, or whatnot. Well let’s find out. If I open up port
8080 on that machine, it’s going to show me the
man behind the curtain, which is Jenkins, an open source
continuous integration service. Quick show of hands, how many
people here have heard Jenkins? Use Jenkins? Oh wow. OK. So many of you. Hopefully you’re
familiar with Jenkins. Well, this is the job that
we’ve created on this machine, aptly titled My Job. I can go to the
configuration of this job. And I’m just going
to quickly scan over all the various things
that we’ve set up. First we do the build, upload
the results, run the tests, Push To App Engine, the
standard sort of fare that you would expect. Now there’s a key point
I’d like to highlight here, which is that we’re using the
Google Cloud Platform to build tools for the Google
Cloud Platform. Not only using Compute
Engine for the build machine, but we’re also taking advantage
of Google Cloud Storage. If I go over to the
Cloud Storage tab, you can see that we create
a Push To Deploy bucket. This bucket stores the results
of build results, test results, and metadata for each
deployment that it does. So these folder
names, these are they Git caches that are being used. And if I open up one of these,
you can see the raw build log. That is the output from Jenkins. The test results, as
well as the actual payload that we’ve deployed. In fact, we use the
metadata for this object to power information on
the Release History page. So I’m going to go
to that real quick. So I can show you how flawlessly
the system has worked, we issued the commit through
the web browser, did the build, ran the test, deployed
it, made Fluttery Bird a worldwide success. Or, it has failed because
the tests have failed. I’m going to pretend
like I’m surprised, but I’ve been working on this
demo for weeks so don’t worry. I can drill into the tests. See, these are the
operations that the build machine has performed. But, of course, the
test have failed. So it’s still within the
Google Developer Console, I can drill into that and see,
well, what actually occurred. As it turns out,
some of the tests passed, but the
high score service did not throw an exception. I just love it when you
have these sort of bugs. You have some bad
logic in your program. So you remove it. And then you
introduced a new bug, which is the unit test that
expected that bad logic now failed. You can’t win them all. Well next I’m going to show
you, using the G Cloud SDK, and go into the
command line and just fixing this using my existing
Git and tools and my existing workflow. But before I do that,
I want to show you one more thing, which is
going back to the Google repo. It’s great that Google
has a Git source code repo that will enable you to do
this Push To Deploy and build experience. But how many people here
aren’t already using Google for their Git hosting? Or maybe using
Bitbucket or GitHub? How many people here use GitHub? All right, similar to
Jenkins, most of you. Which is great. Well, we’ve launched a feature
called connected repos. That is, a way to mirror
an existing GitHub repo and have that transparently
and seamlessly work with the tooling
that we’ve provided. So you can use your Git
repo just like you do today, but whenever you do a
push without needing to add, configure
the remote manually, it will just
automatically trigger these sorts of workflows. So to give you an example,
I have a connected repo, the App Engine
Python helloworld. And I’m going to use the GitHub
based editor to just change it. It prints, “Hello, World!”
and the word orange. Why don’t we change
that to apple? Oh, I guess first, this
is the current state of the application,
swiftcatfish511. You can see it prints orange. Just so you know. But I’m going to
change that to apple, scroll down and– all right. And then I’m going to make
this commit directly in GitHub. But you could
imagine, this is you pushing to your Git
repository like you also do. Shortly, this is going to
trigger the Push To Deploy experience that we’ve expected. And then we’ll hit refresh
on swiftcatfish511, and it will print
the word apple. But first, let me
hand it back to Rae. RAE WANG: Thank you, Chris. And can we go back
to the slides please. I want to show you again
the flow of the demo that you just saw. You can make a code
commit into either the Google hosted Git repo
our your own GitHub repo. So nothing needs to change. Just use the same
commands I used today. The commit triggers Push To
Deploy, which then, in turn, invokes Jenkins to run through
the build test and deploy steps for application. The statuses’ results are
displayed through IU and CLI. And we also preserve high
fidelity logs and put them into your Google
Cloud Storage bucket. So these are features
that you just saw. We know that to help your
releases and deployment to be fast and reliable,
automation is the best way. There are some things
that machines just do better than us, right? Such as repetitive
release tasks. So let’s dedicate to them. For that, we have
launched Push To Deploy to automate the build,
test, deploy steps for you. But we also know
that many of you need to use a number of
different environments before your code hits
production to validate that it’s working correctly. Having to push to invidivual
environments is a hassle. So we are now working
on expanding this to a full multistage pipeline
with permission and quality gates between them. To give you visibility into
your deployments, our UI and CLI tools offer you notifications,
statuses, standard outs, a history of all your past
builds and deployments. But more importantly, linking
them back into code commits. To meet you where
you are and help you to customize how
to do your releases, we have built Push To
Deploy on top of Jenkins, that so many of you already use
and are already familiar with. I just want to emphasize that
we take the hard work of hosting Jenkins so you have to
worry about setting up, monitoring, maintenance,
other annoying stuff. It’s on us. So you can focus on the
more important stuff, which is the application. In the current version
of Push To Deploy, you can select which language
your application wants to use, which, in turn,
selects the number of tasks to run through for
Push To Deploy. And going forward, we want to
make this a more open platform so you can specify
your configurations, customize your build
and test parameters, and even bring your
own Jenkins plugin. So that was deployment,
and that was code. We saw lots of great features in
the UI and the Google console. But I do know for some
of you that command line is what you want to
use on a daily basis. So who are here with me? Command line users? Yeah, a lot of you, right? So you’re going to love it. Next let’s talk about the Google
Cloud SDK and command line tools. So when we think
about command line, we think of fast and
light ways to do things. And therefore, everything
around the command line tool should be fast and easy. And that means it’s
easy to set up, it’s easy to maintain for
any environment that you’re working, and for any member
of your development team. It should also be easy to
learn, to ramp up to use. Especially if you already
use our UI or API, learning command line
should be straightforward. And if the command line is
how you want to do things, it should not only work
for one or two commands. It really should be
available for all the things you want to do on Google cloud. So now we’re going
to go back to demos, and Chris is going
to fix his unit test. CHRIS SMITH: So just to keep
me honest about the GitHub connected repos, I’m going
to refresh swiftcatfish. Just like we
expected, the Git push was all the way through
our system and deployed. All right, so let’s go back
to Fluttery Bird, the world’s greatest app, I might add. So we have a problem. The build works, but we
need to fix this unit test. So again, what’s the
quickest way to do that. Well, I showed you
the web based editor. Now I’ll show you
the G Cloud SDK. So I can do this on
the command line. So you’ve probably
seen this before. But how do you install
the G Cloud SDK? Here’s a quick cheat sheet. Curl, crazy long
URL, pipe to bash. That’s all you need. All right. I’ve already installed the
G Cloud SDK on this machine. So if I wanted to know how to
use it, I could just type help. OK? And that will list
all the things that I can do in sprockets. But I’d like to remind
you all of something. It is the year 2014. We live in the future. And although we don’t
have flying cars yet, I think we can expect a
few other nice features from our tools. So I’m just going to type G
Cloud and press Tab twice. And now we finally have
auto completion right in our console. So if I want to know what I
can do with the components that I have installed from
the G Cloud SDK, G Cloud Comp, Tab will auto complete
to components. And those are the operations. And then finally– Oh, G Cloud. And I can list all the
things that I have installed. OK. Great. Great. Well, one problem
that you’ll always run into when you’re
using command line tools, such as the previous
generation of the App Engine SDK is, how do you
authenticate yourself? Well, you could break
your flow and just have to type in your
password every few minutes. Or, you could set up an
application specific password, but there’s some
security concerns there. Or, you could use
OAuth, and that’s great, but then you have
to authorize it for every single
different resource. For using gcutil, for
pushing to App Engine, maybe Cloud storage as well. Well, I’m happy to talk
about how the G Cloud SDK has solved that
with one command. If I want to authenticate and
log in with all my command line tools as me, all I need
to type is gcloud auth login. This is going to take me
through a single OAuth flow so I can authorize the G Cloud
SDK to perform all the cloud operations for the Google Cloud
Platform on the behalf of moi. So I click Accept,
and I’m good to go. But not only that, I can
type gcloud auth list and show all the
different accounts that I have activated
with the SDK. So if you use two different
Google accounts, say one for managing your
prod environments, or one for managing your
dev and test environments, you can seamlessly
switch between the two all with a couple of commands. OK. So enough about G Cloud. Let’s get back to
fixing Fluttery Bird. Google has this the
Git repo in the cloud. How do I bring that
to my local machine? I could go back to the web,
find the right URL, Git clone, set it all up myself. Or, I can just type
gcloud init fluttery-bird, where fluttery-bird
is the project name. This is going to automatically
bring all the context and set up my Git repository. And so it’s ready to go for
me to make local changes. I’m going to change directories. So here’s all the
source code that I have. In fact, if I type git log,
you can even see the commit that I made on stage
a few minutes ago. And so from here, I can just
use the tools that I’m used to. For example, the world’s
greatest operating system. I mean text editor, Emacs. And I can finally just
get rid of that unit test. And then I can use the standard
Git workflow than I’m used to. And so to fix Fluttery
Bird once and for all, trigger the Cloud build,
Cloud test, and deployment, I end it all with a Git push. So while I’m super excited
about the developer tools that we’re building
and showing today, as well as the ways in which it
can make you more productive, the thing that excites me the
most is that we’re just getting started. And with that, I’ll
give it to Rae. RAE WANG: To bring it
home, here are the features that you just saw. To give you an easy
getting started experience, the SDK can be installed
by a single command line. The component manager lets
you see which components are being installed and
add new ones as needed. You no longer have
to worry about, do I have the latest
version of SDK or not? Because we have auto
update notifications that will keep things
up to date for you. I love G Cloud,
but I have to tell, my favorite command
there is G Cloud init. You saw how easily
it provisioned the entire workspace
for any product you want to work on in just
a matter of seconds. We want to make command line
tool easy for you to use. We don’t want you to have
to memorize and spell any of the other commands. So we provide you
with autocompletion. You can just Tab,
and it’s there. There’s unified help in a
tool that allows you to easily discover and learn new
commands as you go. And we also make a
big effort to try to keep command line,
UI, and API consistent so you only have
to learn it once. In a single SDK, you get all
the tools for Google Cloud. Some of you– I saw
a show of hands– might have used gcutil
or AppConfig in the past. Right? Remember, you had to download
them separately, keep them up to date, install them,
learn them separately. Well that was the past. Now they’re all
coming to G Cloud. So there’s only one tool
you need to care about. Now in the last 40, 45 minutes,
you saw all the dev tools that we have built this far
for Google Cloud Platform. You remember the browser
based code editor diffing tool to log to source linking,
Push To Deploy, and all in one Cloud SDK. These are just our first steps. We’re going to continue to
be relentless in our quest to give you the Dev Ops
experience that you deserve. Thank you. And with that, we’re happy
to take any questions. So you also have my
email there if you want to reach out and
ask questions offline. You can send your email. AUDIENCE MEMBER 1: How about
the GitHub integration? It was announced
this morning but I don’t think it’s available yet. So using GitHUb instead
of the Google repo. CHRIS SMITH: Oh, yeah. It’s there. We’ll be launching it, I think,
publicly in a couple weeks. Or, rolling it out
wider in a couple weeks. AUDIENCE MEMBER 1: OK. RAE WANG: So most of
the demos that you just saw, if they’re not already in
production, they’re currently in our Canary environment. They’re like hot off the
presses, on their way, rolling out in the
next few weeks. CHRIS SMITH: You may have saw
a link to– on Brad Abrams talk about the Trusted Tester
program at Google. That’s how you can get a sneak
peek at some of the features that we’re working on. AUDIENCE MEMBER 2: In
the Git repository, is there a way to sandbox–
if you invite a developer to sandbox the access to
certain portions of the project? Or do they automatically
just get access to the entire project? CHRIS SMITH: Our
Git setup currently does not support that. The ACLs for the repo,
as in who can access it, are tied to the project. So people who have edit
access to that project can modify any part of
the Git source repo. OK. RAE WANG: And there’s
a question here. CHRIS SMITH: There’s the–
Let’s follow the mic. AUDIENCE MEMBER 3: Hey,
I’m just really curious. You talked earlier that
you have one source tree. But the tooling that
you’re generating now seems to deal with if
you have, say, one repo, it goes to one instance. So it seems like
different concepts. And I’ve always done
it the more repo way, but are you working on,
if you have a giant source tree, push out of
this branch of it. Or how do they relate
those concepts? RAE WANG: So that’s a
really good question. He’s talking about
within Google, we have a single repo
as Melody talked about. But we understand that people
do these things differently. There’s some companies that
like to have just a single repo. There’s some companies that
like to have– we have customers with a few hundred
repos on GitHub. And that’s where we
think flexibility is really important. We want to be able to
support these different modes of working. And we also want to make
sure that the benefits on both sides– we can
give it to both camps. For example, one benefit
of having one repo is that it’s easy to
search across them. And it’s also easy to reference
different parts of things. And when there’s an
issue in common library, it’s fixed for everybody. But we think these
benefits can be true, even if your preferred
way of working is having multiple repos. And that’s something
we’re working on. AUDIENCE MEMBER 4: So
do you guys currently offer any way to speed
up the tests themselves? Do you guys do any
kind of parallelism in what you guys are offering? Or is it basically
just hosted Jenkins? CHRIS SMITH: Currently,
we don’t have any tooling about
speeding up tests. It’s just as quickly
as you can configure the Jenkins master
to execute them. But that is a problem
that’s on our radar. I think we have time for
one or two more questions. RAE WANG: And I also want
to add that sometimes it means allowing you to
be– so for example, be able to choose which size
of VM to host your Jenkins depends on how
much your workload. And that’s definitely
something we’re looking at. Yes, one more there. AUDIENCE MEMBER 5: [INAUDIBLE] RAE WANG: So the question
is, do all the command line tools run Windows as well? And actually have an expert here
who can answer that question. [INAUDIBLE], do you
want to comment on– MALE SPEAKER: Yes. RAE WANG: Yes. The answer is simple. Yes. CHRIS SMITH: All right. MALE SPEAKER 2: That’s
all we have time for . CHRIS SMITH: All right. Great. RAE WANG: OK. All right. Thank you.

4 thoughts on “Google Cloud Platform Live: DevOps at Google Speed & Tools for You

Leave a Reply

Your email address will not be published. Required fields are marked *