Joe Conley Tagged cto

Data Visualization Roundup

Wed, 31 May 2023 00:00:00 +0000

Teams operate best when they’re focused. Not just on a shared mission, but on the steps needed to get there. If your team is striving to build effectively at scale (and are largely remote to boot), chances are you’ll need some shared dashboard to check progress and monitor what matters.

I’ve used two such tools extensively so far (QuickSight and Superset), and I was surprised to find how easy both were to use for someone as design-challenged as myself. Herein lies a quick review of their functionality (both work well!).

Intro - SEC data-driven portfolio tracker

As a motivating example, I’m using a few simple financial datasets to show off each tool’s capabilities. EDGAR produces quarterly data dumps of all publicly-traded companies in a TSV format. Companies must tag specific fields (e.g. NetIncomeLoss). I added my own portfolio activity and some historical daily price data to build a dashboard showing financial performance, price tracking, and portfolio composition. This afforded me some diversity of chart visuals and non-trivial datasets to work with. Here’s how the tools compare on some core data visualization concepts.

Working with Data - QuickSight (by a hair)

It became clear that with both tools, it’s always simpler to clean and combine data beforehand. While these tools support joins and other ways to massage data, I saved much time by just passing it clean data (reminded me of how to design tables in DynamoDB - think about queries in advance). However, if you choose to do more customization in the tools themselves, you can use custom SQL views (ideally at the source - JDBC for Superset or SQL/Athena for QuickSight).

Caching data seems to be simpler in QuickSight, specifically in relying on SPICE to store a cached version of your data. Superset has this capability using Redis, but this adds a bit more complexity (especially as you are likely hosting Superset on your own).

QuickSight’s come a long way over the past few years, and the biggest techdiff it has is its seamless integration with the AWS ecosystem and some other interesting third-party integrations as well (e.g. Twitter, Github). Despite the slight edge, I didn’t find much difficulty in managing data in Superset, and could use a myriad of ways (simple script, Airflow, hell even dbt) to prep data for visualization in either.

Building Charts - Superset (by a nose)

Both tools offer a wide and growing set of visualizations to use, but I was really impressed to see the full list of Superset plugins (even as far as Mapbox plugins and Handlebars for templating). This, for me, made up for the sometimes clunky interface for building charts. The biggest gripe I still have is that updating a custom dataset after building a chart won’t reflect those changes in the chart builder (you can still manually reference any new fields but this wasn’t obvious).

QuickSight works well and offers a wide set of visuals to choose from, and as you can see it was easy to replicate the work from Superset to QuickSight.

Superset

QuickSight

Embedded Visuals - Tie

Both tools take a bit of finagling to get the initial embedding working, but once up and running it works pretty seamlessly. Not much of a difference here, except that Superset doesn’t have a way to have visual-specific filters on a dashboard (the filters are only global), but that can be worked around pretty easily (SS embed, QS embed)

Measure What Matters

Modern BI tools have come a long way. While there’s dozens to choose from, you can’t go wrong with these two. If you’re cost conscious and not married to AWS, Superset should work great. If your heavy AWS users then you probably won’t mind the incremental cost of a few QuickSight seats given how fast you can design and create its easy-to-use visuals.

Complexity A-Twitter

Fri, 23 Dec 2022 00:00:00 +0000

As a software executive and engineer, it was fascinating to hear Elon’s chat with the Twitter team about the tech stack:

I really appreciated hearing the intense focus on customer ROI. This is clearly one of the key constraints that the new team is operating under in their mission to get to breakeven (Twitter currently loses ~3.5B a year). This is one of the things I admire most about Elon: his hyper-practicality. Why would an advertiser spend marketing dollars(specifically in the face of a recession) if they didn’t have a clear idea of the ROI? Between that and cost-cutting initiatives, it seems like Twitter is on the track to a more focused, disciplined operating structure.

What struck me the most though is the focus on removing complexity from the software stack. Removing complexity (i.e. paying down tech debt) is not something you can tackle every few years, but has to be a constant vigilence. Sure there are times (especially during your startup phase) where you choose to incur this debt in the face of customer demands or budgetary constraints. But like all forms of debt, interest accrues. If you don’t pay it down you could be left with a stack like Twitter’s, replete with 17 different microservices that while decoupled still have significant implicit dependencies, making any code change non-trivial.

Another symptom of excess complexity is onboarding time: how long does it take a new hire to ramp up and commit a small code change? The Twitter team mentions that today it takes at least a few weeks for a new hire to get setup and productive, which in today’s day and age seems like an eternity. Between IaC, Docker, and other automation tools, onboarding ramp-up time should take days, not weeks. (N.B. I’m just as guilty of giving onboarding/documentation short shrift, but as a tech executive what could be more important than keeping your team productive and thereby happy?)

I’m excited to see what Twitter can do over the next 18 months. It does seem like a few recent changes has improved performance. Especially as, like Elon said, “we’re not splitting the atom here”. Compared to his other endeavors this should be a cakewalk!

Orwell's Rules for Writing (Software)

Fri, 26 Feb 2021 00:00:00 +0000

I’m in the middle of reading Churchill and Orwell: A Fight for Freedom and so far really enjoying the parallels drawn between two of history’s most important figures. Both had military experiences which informed their worldviews, both suffered serious career setbacks, and both keenly saw the threat of totalitarianism for what it was and worked tirelessly to prevent its spread.

About midway through the book the author enumerates Orwell’s six rules for writing. Examples follow of Churchill having similar thoughts on the brevity, economics, and power of language. So I thought I’d help translate these ideas for any software engineer who might care to listen:

1. Never use a metaphor, simile, or other figure of speech which you are used to seeing in print.

Never use common variable names like data. That’s not very meaningful, all of this stuff is data! Metaphors and simile don’t tend to be useful unless the ideas are truly core to your abstraction.

2. Never use a long word where a short one will do.

Be thoughtful when you name things. Variables, functions, and classes should have the shortest and most meaningful names possible. You’re building a mental world from scratch, and the language you choose defines how everyone else will think.

Conversely, unless your indexing over a list, single-letter variables are probably too short. Seeing variables a through z littered about code without any context or meaning is plain mean. In my early days of engineering, I heard stories about a paranoid contractor who intentionally used obscure naming to make his code impossible to read, thus making him “indispensable”. Fool of a Took! It didn’t work, and I think both Orwell and Churchill would agree there’s a special rung in Hell reserved both for people who waste good scotch and engineers who use language to such evil ends.

3. If it is possible to cut a word out, always cut it out.

Prefer functional methods and languages where possible, i.e. choose Scala over Java. This can dramatically reduce lines of code, and help you think at a higher level of abstraction that is generally a positive trait for folks who live and create worlds of abstraction most of the day. Yes imperative code might seem easier to write but if you have to do anything at scale you need tools like this in your toolbelt.

4. Never use the passive where you can use the active.

Have clear separations of concerns in how you architect systems. If a single method does a dozen different tasks, refactor it. This empowers your objects to have clear and distinct jobs and focus on doing them well. This allows to build high-performing systems at scale if you can reliably compose modules that do what they’re supposed to do.

5. Never use a foreign phrase, a scientific word, or a jargon word if you can think of an everyday English equivalent.

When you talk to the business folks, speak in simple terms. Trust me, they know you’re smart, you don’t need to show off. Speaking in plain English helps you show that you’re more than a code monkey, and helps them learn more about software and feel empowered about the product.

6. Break any of these rules sooner than say anything outright barbarous.

If your startup’s MVP needs to work for tomorrow’s demo, then yes by all means write your imperative for-loops and mutate state to your heart’s content. But be mindful when the exception starts to become the rule.

Hire Joel Embiid

Fri, 15 Jan 2021 00:00:00 +0000

Startups are a team sport. The players you choose to work with have such an outsized impact on the team (for good or ill) that you really have to have “A” players at every position. But it’s not enough to just say “go hire all-stars”. There’s very specific attributes you need to look for, and that I found this week watching Joel Embiid lead the Sixers to a gritty overtime win.

Joel Embiid is the embodiment of the Sixers philosophy in hiring (AKA drafting) championed by Sam Hinkie. He has grown to become a dominant force in the league, consistently ranking at the top in scoring and defense. But when you see him play it’s clear he’s not just your typical all-star. He has a unique blend of skills that serves as a powerful model for early-stage startup hiring.

Eye for Talent

Source: https://www.youtube.com/watch?v=faZY6W3toh0

Pictures speak volumes. Beyond his pure physical growth, Embiid is improving his game every year. He’s consistently added different nuances to his offense. This year he’s improved his recognition and passing out of double teams (helps to have shooters to pass to, thanks Daryl!). This eye towards growth has translated into success on the court.

Startup hiring is about finding young talent that you can grow into high performers. It’s probably the trickiest, most challenging, and arguably the most fun aspect of hiring. Like Billy Beane trying to find inefficiencies in the market, it’s rewarding to not only identify talent early but help them grow as a significant part of a team trying to build something from nothing.

What’s even more poignant about startups is you have to get it right fast! Pro teams get a new draft class every year, and can make up for mistakes like this or this or even this (though I’m still rooting for him). Startups don’t have that luxury. The window for getting the timing right can be so small, and can literally mean life or death, that you have to be brutally efficient when it comes to hiring.

Wears Many Hats

Here’s the line for Embiid from Tuesday night:

Source: https://www.espn.com/nba/boxscore?gameId=401267317

Excellent scoring, great defense, made all 13 of his free throws, just all-around dominance. This is a big man who can be top 5 in scoring and defense, is strong from the foul line so you can’t resort to hack-a-Shaq, and is a good enough shooter to force defenses to respect his presence beyond the arc. At one point on Tuesday he had three guys trying to defend him to no avail, I mean how do you stop a force like that?

Point being, it’s very difficult to build startups with specialists. You are going to pivot at some point, and the cost of having high turnover to constantly mix and match narrow puzzle pieces is too inefficient. Seek generalists early, then once you’ve established traction think about more specialized roles. Generalists cost too much you say? Then be generous with the equity upside. Give them real skin in the game, and then give them room to operate.

Pure Will

The most impressive feat in Tuesday’s win was in the third quarter, Embiid dropped 22 points in 8 minutes. He literally decided to take over the game, got the ball almost every trip down the court, and willed the team to victory. Talent is one thing, but having that pure strength of will is rare and really fun to watch.

Find me a team of folks who have that same will to not only push through on execution but also the honest determination to understand what’s true and the vision to navigate, and I’ll bet on them every day of the week. It takes a special strength of character to survive the inevitable ups and downs of startup life.

Culture Matters

I’ve been a Philly sports fan all my life, and Embiid has to be near the top as far as local popularity goes. He’s outspoken, goofy, works hard and wants to win. Similar to Allen Iverson, he gives it his all, and expects his teammates to do the same.

It’s cliche to say that culture matters in startups, but it’s important to note that it starts from the top. Leaders set the culture, either consciously or otherwise. Now more than ever, it’s important to take care of your team, understand their challenges on and off the field, and build an environment that supports them in every way shape and form. Who wouldn’t want to work for a team with a culture of excellence?

Trust the Process and Have Fun!!!

Lastly and most importantly, we’re on this earth a short while. Let’s have fun and play hard!

What is a CTO?

Tue, 01 Sep 2020 00:00:00 +0000

I’ve spent a lot of time thinking about what a CTO is. It’s important to really think through this role as it can have such a huge impact on companies both large and small. You can search the web for “startup CTO vs. big company CTO”, “CTO vs. VP Engineering”, etc., but those debates miss the point. What matters is understanding the function of a CTO in your organization. And I assert that a CTO is an executive who serves the business through the deployment of technology.

That’s it! The details of implementation will vary widely based on company, industry, and the strengths and weaknesses of the individual CTO. But at its heart, this role requires a leader who has a strong fundamental understanding of technology and how to use it to support a business.

What, then, is technology? It’s easy to assume that things just “get better” over time. We’ve emerged from the cave and tilled the earth, dammed the streams, and harnessed the power of the sun as we march towards greater standard of living. However, none of these advancements were inevitable. Progress depends on us inventing the future.

This topic is more important than ever. It’s hard to argue that software is eating the world, and as a result, every company is now becoming software. Technology advancement has accelerated change in every walk of life. Powered by trends of cheap computing power and new techniques to analyze data, this wave of extracting value from data is overhauling every and any domain it gets its hands on.

Is this wave of technological progress “good”? Any type of significant change can and will cause issues like job displacement, but I believe that as technology helps automate mundane labor, it should create opportunities for human beings to focus on higher-level, more creative work. This underscores the importance of having strong, thoughtful education for everyone young and old.

Given the speed and complexity of these trends, it’s important to have someone who understands technology at a sufficiently deep level (software, hardware, whatever). I’ve found the best CTO’s form strong opinions about their technology choices, yet dispassionately remain open to searching for the best solution. They rely on tried-and-true systems rather than chase what’s trendy. They push hard for excellence, but balance that with sensitivity to solving the customer’s problem in a cost-effective way.

It’s also important to note that a CTO is an executive. I think the toughest thing for a newly-dubbed CTO to learn is how to scale not just code but an entire organization. As a lead engineer or architect, your choices are mostly constrained to scaling infrastructure or analyzing data. Once that “C” shows up in your title, however, your biggest challenges tend to be people-problems. Which seats on the bus do I fill first? Can I grow this junior resource into a contributor, or do we need to move on? Can I make peace between two warring factions? You’re definitely moving “up the stack” here, and it requires a wholly different skillset than what got you here.

Personally, I value the opportunities I’ve had to serve as CTO because I know that I can have a meaningful impact on a company. I know my decisions will have consequences, and I relish the challenge to move the needle where I can. Most of all I enjoy aligning a team towards a common goal. By its nature, technology strives to improve how we live our lives, and to serve as the steward for its application is a privilege.

I’ve also been blessed to have found an informal network of mentors who I rely on regularly when facing tough decisions. I’m hoping I can start to do the same.

At the end of the day, it’s just really cool to helping spearhead the deployment of technology for an organization. I’m a huge Elon Musk fan for a lot of reasons, but his quote about why he went into engineering really spoke to me:

Just really cool to think about engineering in terms of building new things which enable really smart people to keep making progress and ultimately improve our collective consciousness. It’s inspiring to think we’re all trying to play a part to create a better future. It’s why I pivoted from a pure mathematics career to one grounded in technology. Because it’s “what’s next”

Shape Up or Ship Out!

Wed, 29 Jul 2020 00:00:00 +0000

Have you worked on a software team that consistently shipped high-quality work on-time? I’ve seen companies large and small have plenty of difficult delivering software. Either things break in production, or things don’t ship on time, or worst-case both.

And in most cases, the antidote was supposedly to “be more agile”. Planning poker, Fiobancci story pointing, backlog grooming, and all of the other well-trodden ceremony should make these issues disappear. And to be fair, I’ve seen these things “work”, but only to a certain extent, and usually over the short-run. I’ve tried them myself with mixed results, and an underlying apprehension I could never put my finger on. Did software engineers pick this profession out of a love of project estimation and ticket management tooling? Does any of this feel like a natural way to treat human beings?

And so it is with this undercurrent of dissatisfaction that I recently found Shape Up by Ryan Singer. It’s a refreshing take on how to craft software. The book is a perfect size to read, and has a great blend of thoughtful ideas paired with practical advice from the trenches of a well-known and successful product company, Basecamp.

This book hit a lot of notes for me, but the biggest idea was how work is shaped and handed off to the development team. Projects aren’t chopped up into tickets, doled out and estimated. The product strategy team defines a project with CLEAR BOUNDARIES and a FIXED TIMELINE, and lets the development team work through the details and manage themselves, with clear checkpoints along the way. This allows developers to use their brains to come up with their own implementation ideas, frees up the product team to stay focused on high-level strategy, and doesn’t require a middle-manager (i.e. Scrum Master) for communication.

This is the essence of management that delegates without abdicating responsibility. Shaping creates guardrails to allow a team to grow and experiment without hand-holding, trusting but verifying that the project’s thoughtful criteria will be met. I think these ideas would be borderline radical in most companies I’ve seen.

Why does any of this matter? Because I believe technology companies can be agents of change. Crafting tools that help automate and streamline rote tasks so human beings can focus more on human work should be the ultimate goal of most technology offerings.

In that same spirit, how do we as technology leaders ensure that the builders of technology are allowed to do more human work? By giving them more responsibility in an environment where they feel safe. I think the true measure of success for any company is not just revenue growth or user adoption but how did your team grow professionally and personally? Otherwise, what’s the point in winning?

In addition to managing teams, I think the concept of shaping and fixed deadlines helps us manage ourselves too. Rather than keeping an iron-clad todo list, think at a higher level what you’d like to accomplish for a given week or month, make sure you focus on achieving the core “functionality”, and then DELIVER. Keeping a blanket list can trick your brain into thinking all tasks are created equal and MUST be done. Being more thoughtful about time constraints and scope helps to reframe and reorient your mind, and start building your shipping muscles.

This speaks to another key point that Singer, Clayton Christensen in Competing Against Luck , and even Peter Thiel have all talked about. Namely, a lot of the time the key to solving a problem is how to frame the question. Whether it’s deciding what features to build into a product, how to build out your organization, or how to spend your life, the way we frame the question helps guide us to the answer.

Ultimately, finding humane ways to ship software will be an ongoing challenge for leaders and contributors alike. Amid the rise of automated coding tools like GPT-3, there will be a need for thoughtful, effective leadership that understands what questions to ask and what tools to build.

So Shape Up, or ship out!

Ask a CTO Interview with Sarah Herrmann

Tue, 31 Mar 2020 00:00:00 +0000

I recently had the pleasure to speak with Sarah Herrmann of Juno Search Partners to kick off her new “Ask a CTO” series. I really enjoyed the chat and I hope my thoughts may be helpful!

https://www.linkedin.com/pulse/ask-cto-joe-conley-sarah-herrmann/

Forbes Technology Council - Encouraging Developer Side Projects

Fri, 13 Dec 2019 00:00:00 +0000

I was recently featured in a Forbes article about encouraging developer side projects.

https://www.forbes.com/sites/forbestechcouncil/2019/07/18/14-tech-pros-share-their-best-tips-for-troubleshooting-code-malfunctions/#52c7cefe4084

Working at Buyside

Thu, 14 Nov 2019 00:00:00 +0000

This post originally appeared on Working with Buyside

Our team is growing! I’ve had the distinct pleasure of building out our engineering team over the past year, and as we’re prepping for another round of growth, I thought this was an appropriate time to reflect on what our culture is like. Why do we enjoy working at Buyside? What makes us tick? I asked my team to reflect on that, and I’ve summed up the main points here so that prospective hires can get a good sense of why they might want to join Buyside.

Autonomy

This was the most common theme by far. Our team is well-distributed geographically, from Pennsylvania to Nevada to Costa Rica, and we pride ourselves in creating an effective environment conducive to remote work. We keep meetings and overhead to a minimum. We subscribe to the adage of hiring smart people to tell us how to do things, and not dictate to them. We encourage healthy debate, and expect team members to take ownership of their work from inception to completion. This allows all of our team members to grow while serving the needs of our clients.

Productivity

Similar to the above, we want to create an environment that optimizes productivity. This isn’t treating our team as cogs in a machine, but rather understanding that when we hire smart and motivated people, they tend to be happiest when they are feeling productive. We provide top-of-the-line machines. We try to minimize context-switching as much as possible (some days are better than others). As most of us have families and kids, we respect work-life balance and allow for flexible schedules. We know this approach is a win-win for both our team and our company.

Value

Despite our recent growth, we still maintain the scrappiness and energy of a startup. We expect everyone to make meaningful contributions to the direction of the company, and I as a tech leader take great pains to ensure the team is constantly aware of the impact of their work. One advantage we’ll always have over the bigger tech companies we compete with over talent is offering the opportunity to be part of a special team that’s offering real tangible value to their clients.

And we’re just getting started! I’m really excited about the work we’re doing now and in the short-term, as the next year promises to be one of unprecedented growth for Buyside. If you think you might be a good fit for our team, please check out our Careers page for our current openings. Thanks!

Managing Distributed Teams

Fri, 01 Nov 2019 00:00:00 +0000

I was recently featured in an article discussing how CTO’s manage remote teams. It’s certainly a non-trivial task, but an increasingly relevant one as software teams become more and more geographically distributed. I think this a great trend, as your potential resource pool becomes much wider once you look beyond your backyard.

I’ve certainly been blessed to have found such strong talent to work with at Buyside, both locally and remotely, and our company is the better for it.

Enjoy!

https://arkenea.com/blog/managing-distributed-teams/

Leadership Aboard the Santa Fe

Thu, 03 Oct 2019 00:00:00 +0000

What is leadership? It’s certainly a fancy term. It adorns countless corporate walls paired with some inspirational message or image. It appears as a common requirement for most management-level job applications. It can even inspire folks to come together in an almost cult-like fashion:

But defining leadership can be tricky. We can use words like humility, decision-making, servant to try to arrive at a definition, but that’s boring and often incomplete.

Leader-Follower to Leader-Leader

I recently read L. David Marquet’s Turn This Ship Around!, and came across a truly great example of what it means to be a leader. In short, leadership is creating an environment that helps your subordinates grow into becoming leaders themselves. Marquet did this in a seemingly unlikely setting: aboard a submarine in the U.S. Navy.

How?

Here are my main takeaways from the book to how to shift to this new paradigm:

Create an environment of ownership

It’s important that the environment is tailored to allow reports to make decisions and think on their own, but be accountable for their decisions. Marquet encouraged this by requiring his junior officers to recommend what actions to take. Instead of Marquet giving orders, he’d have his reports make decisions, having them say “I intend to” before acting. e.g. “I intend to take the ship up to periscope depth…”, while Marquet would give simple confirmation. Even little things like language can help encourage passive officers to think independently.

Delegate but don’t abdicate

Marquet tried out a Leader-Leader experiment on the U.S.S. Will Rogers without much thought into how it was implemented. This led to an oversight in readiness, a failed inspection, and a quick transfer to a Soviet outpost where he could spend time reflecting on just how poorly his first efforts had been. Simply delegating tasks without thought to how they’d be carried out is a recipe for failure.

Once he got another shot aboard the U.S.S. Santa Fe, he focused more on developing the level of competency of his officers. He resisted the urge to dictate solutions to problems, but instead gave his folks latitude to work through issues. As Marquet succinctly put it, “Don’t brief, certify”.

Communicate vision without dictating

Communication is paramount. Especially for significant changes that are new or uncomfortable, you can never communicate too much what your vision is.

Applied to Tech

I’ve certainly struggled with delegating, especially on a small team where there’s plenty of work to go around and never enough bodies to do the work. I think I’ve gotten better at communicating context and the necessary details of a particular issue or task, however I know sometimes I’ll lead with my proposed solution without allowing my reports to come up with their own design.

But overall the ideas in this book have helped me start to think more about how I operate and how I can do a better job of creating an environment that creates great leaders.

Results

Marquet’s work resulted in the Santa Fe earning a record number of awards in both excellence and most improved. Furthermore, Marquet’s officers were promoted at unusually high rates to lead their own ships, and applied most of what they learned aboard the Santa Fe to their own ships.

In business, it’s easy to get caught up in optimizing specific metrics, growing profits or increasing market share. But doesn’t that all pale in comparison to helping others grow? To inspiring others to build a brighter future? In that way, leadership can be boiled down to priorities. How you choose to spend your time, what you choose to value above all, and how you choose to impart those beliefs to others.

How do you define leadership?

RVNTV Interview

Tue, 24 Sep 2019 00:00:00 +0000

I did an interview recently on Tech Talks, a local tech show hosted by Michael Bertoni. I talk a bit about my time at Buyside, how I’ve built my teams, and what I’m looking for next in ML/AI trends. Enjoy!

https://drive.google.com/file/d/1XcL-ZGplQmzrdS70mXpiM92QGscRC_sI/view?usp=sharing

Forbes Technology Council - Troubleshooting Code Malfunctions

Thu, 18 Jul 2019 00:00:00 +0000

I was recently featured in a Forbes article about troubleshooting code. Enjoy!

https://www.forbes.com/sites/forbestechcouncil/2019/07/18/14-tech-pros-share-their-best-tips-for-troubleshooting-code-malfunctions/#52c7cefe4084