Dan Antion

Geeky Stuff – I Know, I know

I want to thank all the people who weighed in on my post yesterday with comments about the Thursday Doors Recap page I am building. I appreciate the time you took to comment. I appreciate your encouragement and most of all, I appreciate the fact that so many of you were concerned about my time. I threatened promised to explain how I am preparing the summary. Of course, there’s a little bit of a backstory.

Note: if you find any typos, The Editor may have fallen asleep while reading this. If it affects you the same way, there will be no hard feelings if you skip to the gallery.

Since taking over the management of Thursday Doors, I have been concerned that people looking for the other doors posts have to wade through a lot of comments. That concern is compounded by the fact that people can contribute to Thursday Doors until noon on Saturday, but I have a regularly scheduled feature on Saturdays.

I was uncomfortable saying, “you can contribute on Saturday morning, but no one is going to see your link unless they start poking around my blog.”

Back in January, I made a list of participants and the URL of their TD post. I liked the list, but it took too long to prepare. I might have the time in the dead of winter, but when it gets warm, I’m not going to want to be assembling this list link by link. I have a subscription to LinkyTools, but my experience with that service is that some people are uncomfortable registering with “one more thing” in order to participate. Posting a link, or a pingback to No Facilities works, people are used to the procedure, why change to provide a feature people might not even want.

I started looking at software solutions for extracting links from a webpage – there are many – most were overkill and left me with one person’s idea of a perfect list. Then I stumbled on a post on Medium by Phil Gorman. He had a little bit of JavaScript (programming language) code that extracts links. In addition, he explained how it worked and where it could be modified. I liked that. I have worked with JavaScript before – it’s not my happy place – but I’m retired, I’m still capable of learning and I enjoy programming. On the other hand, I needed to know that a) this would work, and b) people would like it.

If you consider a blog page, with its comments and sidebar widgets, it might not surprise you that there are over a thousand links on that page. I modified the JavaScript to extract the links from only the comments section of my blog. That sounds good, but between links to avatars, blog posts, Like buttons, Reply buttons, Edit (because it’s my blog) buttons and the link to the comment itself, there are still a few hundred links and only two or three dozen matter. Of course, I can filter those links.

Since I am in the proof-of-concept stage, I decided to move the raw list into Excel. I heard several of you groan. I will spare you the details, but these are the steps Excel helps me execute:

  1. I delete all of my reply comments and the comments without links
  2. I delete the links to everything that’s not a comment
  3. I extract the Domain name (i.e. NoFacilities.com) from the URL of the blog post being added
  4. I collect the title from links added by Pingback
  5. I match the domain name from step three with a list of “friendly names” I created for those domains
  6. I create a hyperlink from the Pingback text or friendly name and the URL of the Doors post
  7. I copy the list of hyperlinks and paste it into the Block Editor (creating a table)

The steps listed above are mostly automatic. Ignoring for the moment the time it took me to do all this. It takes about 10 minutes from the time I extract the links to the time I paste the table into WordPress. I can live with that. If I move more of the process to the JavaScript program, I can probably reduce that to five minutes.

I am going to leave this technical kludge in place for a few weeks. One think about running the process in Excel, is I get to see the things that send it off the rails and work out a way to tweak the process steps to accommodate those things.

On Thursday, I will have a survey with a few questions for Thursday Doors participants, the results of which should help me improve this list and the process. I’ll also have some doors from downtown Waterbury, Connecticut. Today’s gallery is just the usual suspects.

Maddie and I ventured out on Sunday.
That is a sheet of ice.
That is mostly ice.
That is all ice. Maddie will jump up onto the snow bank, but it won’t hold me.
Unlike the MICROSpikes we wear in the yard, these are NanoSpikes for walking and jogging. Perfect on ice.
These two guys are hiding in a crevice. I don’t blame them.
Maddie is happy to be back in her park.
It was 22°f (-6°c), less with the wind chill but it was good to see her waving in the breeze.
Our yard remains a mix of snow and a couple inches of ice.
I love this little weed. She decided to pop her head out of the snow next to my shed to see what’s going on.
Smokey heard me crunching through the ice.
The Editor calls this one Arnold. He’s the only one that stands up on his back legs like this.
Small stream draining into the Connecticut River.
One of my favorite little streams.
Winter is the best time to view the Connecticut River from this road.
“Don’t touch my foot”

77 comments

  1. Hi Dan – I noted … but loved the photos more … Arnold looks like he’s got tummy ache?! Must not disturb Mimi or MUmu – let them be! Those spikes are interesting to see … not common for us … all the best through the cold days – Hilary

    Liked by 1 person

    Reply

  3. I loved the geeky discussion, Dan, though your technical skills far exceed mine. I spend most of my working life as an analyst. When I retired as a full-time government guy and came back as a contractor, I started working with servers and scripts and such, which scare me, because I suddenly had the power to really mess the data up if I was not careful. I remember telling a colleague that I thought I was turning into a geek and he looked at me, smiled, and replied, “Turning into…?” I love the long shadows, as always, and the view of the flag. Thanks for the intro to Arnold–he looks like a cool squirrel.

    Liked by 1 person

    Reply

    • I started our as a systems analyst, but I kept getting dragged into programming. I do enjoy it, as long as it’s not an accounting system. It’s fine to let your inner geek out no and then, Mike. As for having the power to really mess the data up, yeah, I’ve done that – cleaning up was a challenge, but we got through it. This is the first good picture I’ve gotten of Arnold. Most stand up with their butt on the ground. This guy looks like one tough hombre.

      Liked by 1 person

      Reply

  4. I know you know I flew right past the geeky stuff Dan! Lol.

    Love the shadow shots of you and Maddie. Smokey looks fit as a fiddle. And now enters Arnold who is too cute for words. He has the “begging” pose down pat. That little guy couldn’t have found a better home.

    I’m so impressed by that weed! What a good role model for all of us.

    “Don’t touch my foot” is still cracking me up. 🤗

    Hope this is the start of a great week for you and the Editor.
    Ginger

    Liked by 1 person

    Reply

    • Thanks Ginger. I knew a lot of people would fly over the geeky stuff. It’s OK. But I didn’t want people to worry about the time it takes me to prepare the list – a few hours of geeky stuff and then it’s easy peasy.

      The Editor spied (and named) Arnold early in 2020, but it’s been hard to get a god picture of him. He’s just so dang cute. I think he knows it, and his “feed me” act always gets results.

      That weed is my fav. I didn’t pull the snow away from the shed because I didn’t want to hurt it. That’s spring spirit if I’ve ever seen it.

      I hope you have a great week, too.

      Like

      Reply

  5. Reminds me of a few projects from my earlier work life when ‘screen scraping’ was a thing. I understood your post and salute you for the research and the results. It is a kind thing you are doing to make spreading the word about Thursday Doors more streamlined. Always love your pics. Spring is just around the corner and it will be a great time to capture all of Mother Nature’s changes.

    Liked by 1 person

    Reply

    • Thanks Maggie. We have a longer run to the corner, but spring is coming. March is usually the point where, even if we get snow, it’s gone in a few days. I remember screen scrapping exchange rates out of the Wall Street Journal online, before they finally relented and agreed to email us the table. The information is all there, it’s just a matter of finding and formatting.

      Liked by 1 person

      Reply

  8. I really did try to understand the geeky stuff. I will happily be part of the survey on Thursday. Loved the photos, especially sunsets, the flag, and of course Maddie. Have Smokey and Sammy welcomed Arnold?

    Liked by 1 person

    Reply

  11. I used to create tables and active links to files on maps for GIS so applaud your efforts. As I said before I would not be likely to click on the links to see other posts unless the table included the thumbnail link of the person who commented plus a link to their door post. As it is now I can scroll through and pick which ones with links that I want to check out at a glance and at the same time see who was posting. Making tables creates a whole lot of extra work for you. For your sake, I still think it is best to leave it alone. It is easy to see who provided a link and for me to skip by comments that don’t include links. I never have time to visit them all anyway. I fail to see how putting the links in a table without seeing who posted the link can be an improvement. I am basing this comment on the table you recently posted.
    One way to streamline links in comments would be to have a rule that after one day comments, though appreciated, but not containing links to door postings will be removed after a response to eliminate clutter. This way only comments with links will be visible on Thursday Doors. This also entails extra work so that is not an ideal solution to the clutter either. Just a thought.

    Liked by 1 person

    Reply

    • The idea of the list seems to be popular. As best as I can. I am forming the link from the person’s pingback or from a friendly name derived from the URL. It isn’t just the URL to the blog post, in many cases it’s the blog title or the author’s name that shows in the list. I’m going to give participants an option to make that name a bit more informative, but that will be something for people to opt into.

      I wouldn’t want to close comments to people not providing a link because that would include the people who are just visiting my blog and commenting on my post.

      This is one of those things that, if enough people like it, I think it’s worth what has become a minimal effort on my part. If people prefer to read through the comments, they will remain just as they are – this is in addition to that and I designed it with the hope of promoting the TD participants.

      Like

      Reply

  12. I love Arnold and the furry kids way more than Monday morning talk of Excel spreadsheets. Good luck with whatever you are doing and have fun being the Thursday door Grandmaster.

    Have a wonderful Monday, Dan!

    Liked by 1 person

    Reply

  15. Arnold is adorable!! Smokey has competition in the ‘cuteness’ category. Oh, you geeky types…. A neighbor and I were talking this weekend cause she is retiring after this week. First thing on her to-do list is get her husband to clean out his mess. She said he has a box full of old laptops, routers, roller-ball mice, etc., that just need throwing out. I told my husband and could almost see his brain dancing with glee as he thought of all the ways he could ‘swap out parts’….which drives me nuts! Sounds like you have a great plan in store for doors, though! Go forth and have fun!

    Liked by 1 person

    Reply

    • Haha – I tossed old laptops, desktops, and all the assorted bits last year. Including a laptop running Windows 2000! I’m sorry I didn’t check with you first.

      Arnold is a little cutie. My wife loved him. I’ll stick to the deployment plan. No more talk about Excel.

      Liked by 1 person

      Reply

  16. Dare I repeat that Arnold is a cutie? I’d be out there with a bag of peanuts for sure. Appreciate you sharing the NanoSpikes. Appreciate your efforts on improving Thursday Doors. I did read the geeky stuff; still not totally convinced the list is necessary (I enjoy reading thru the comments …) but it also sounds like you’re able to make the computer do most of the work … and as long as it stays fun for you …

    Liked by 1 person

    Reply

    • You can get on board the Arnold fan wagon. My wife is enjoying her buddy getting a better reception than my geek talk.

      The list isn’t everyone’s cup of tea, but if it works for some and it’s easy to do, I’ll keep up with it.

      The nano spikes are great on days like this. The dog wants to go for a walk, but there are patches of ice everywhere. The snow melts, has nowhere to go and then freezes.

      Thanks.

      Liked by 1 person

      Reply

  17. I’ve always found Excel to be useful — and in unexpected ways. That’s where I keep a “character matrix” for each of my stories. I include a lot more than just the characters. You went to a lot of work, Dan. I’m sure the results pay off.
    Tee-hee, I love “Arnold” and that foot closeup. Thanks for the photos. Stay safe and well. Hugs on the wing!

    Liked by 1 person

    Reply

    • I remember you talking about how you use Excel for a character matrix. That’s a really good idea. It is a great tool. I have a better tool, a programming environment in which I could make this a dirt simple drill, but I can only run that on one laptop. If I keep this with a little JavaScript and a little Excel, I can do it anywhere, on any device I happen to have. In many ways, I learned that that kind of convenience is better than a slightly faster program.

      Arnold is winning over hearts today.
      Hugs!

      Liked by 1 person

      Reply

  18. Dan I admire your tenacity and keeping the hinges working on Thursday Doors. I hope WP does not do anything to throw a monkey wrench into your code. When you started describing all of the places links show up in a page all I could do was nod in commiseration. Will your routine for harvesting the links be effected by ‘slow’ internet ? Not that we ever run into that. The WP happiness engineers have had all sorts of problems with their lists over time. The most recent being some sort of truncation. I use the personal lists in the wp reader to follow blogs. There were a couple days this past week that the reader pulled back about 20 blogs and then just stopped like someone had truncated the size of the table. Like I said I hope the happiness engineers don’t do anything different to surprise you. If it were Saturday I would pick up the tab on the next round. Cheers.

    Liked by 1 person

    Reply

    • I don’t think speed will affect the process, John, except for the fact that the whole page needs to be loaded. Once it’s loaded, the JavaScript runs in the browser’s console screen, and it’s pretty fast. The only thing WP can do to mess it up is change the nature of their static links (Reply, edit, that sort of thing). I wouldn’t put it past them to do that, so this will never be a set-it-and-forget-it kind of operation. I need to review the results.

      Liked by 1 person

      Reply

  19. You do what you geekily do, Dan, and I’ll live with the results and, I imagine, be happy. As long as TD continues, I’ll be good. But I understand what you mean about going through comments. Even if a person likes to read the comments, which i often do, it takes a lot of time and they might like to more easily access the links the way you’re showing. At any rate, I enjoyed the photos and enjoy sharing with you that our high today is supposed to be 77. Yikes! Hitting the floor running now.

    janet

    Liked by 1 person

    Reply

    • 77? Wow. We’ve had 7, 17 and 27, but the possibility that the 7 will be first in a 2-digit number…unlikely for a while.

      Working with the code and Excel was a fun project. I’m making a short list of things to change. Once I have enough of those, I’ll spend some more time with it. I’m not changing the way the comments work, but I hope to give people options. I think the list is a great way to catch up, especially with the people who paste later in the process.

      Liked by 1 person

      Reply

        • I’m glad you like that, Janet. Again, once I set that up, it’s a trivial thing to create the link each week.

          I will share a funny story. It took me two weeks before I realized that I just had to schedule my Thursday Doors post, I didn’t have to have it 100% ready to read on Tuesday night. All I really need is a title ;-)

          Liked by 1 person

          Reply

  21. Aha,  Arnold proves the reincarnation theory. . .I wonder who he was in his past life?  We’ve had high winds gusting through the nights which doesn’t make for peaceful sleeping.  Today it’s in high thirties which would be pleasant if not for wind!🌬

    Sent from Yahoo Mail for iPad

    Liked by 1 person

    Reply

  24. Arnold is cute, Dan. I have to start naming the creatures in my garden. I’m not a geek and you took 10 minutes to do what you did is amazing! I used to participate in Thursday Door long time ago. I ran out of doors!

    Liked by 1 person

    Reply

  25. I like the idea of a kludge for a few weeks. It gives you time to examine cases and build a better logic. Of course, it is your time, and you need to decide how much of it you want to spend.

    My problem with such a process is the following, A year later a new special case will come along. By them I’ll have forgotten what I did (perhaps even the language I wrote in) and I will be too lazy to go through it again. I hope you can organize it better for yourself.

    Liked by 1 person

    Reply

    • I am smiling at your future scenario, it is possible. I will try to keep track of the little things that run this off into a field. I’ve made notes about why I did what I did, and I may leave it as a kludge.

      Liked by 1 person

      Reply

  28. According to what Firefox has sent me, their browser doesn’t recognize java. They don’t use it. Is this going to be a problem with your Thursday Doors project?

    And just to let you know, the temperature reached 61 today (Tuesday). :P

    Liked by 1 person

    Reply

    • I only need to run the JavaScrip program on my end. It runs well in the Firefox console, Once the list is prepared, it is formatted by WordPress as a standard blog page. I have viewed it in Firefox, Chrome, Safari and Microsoft Edge.

      61? Wow – we got up to 39 and it felt like spring.

      Liked by 1 person

      Reply

  30. Sounds like you have it down to a science. My approach is to have a local copy of WordPress running under MAMP (a prepackaged LAMP stack for Mac). Each week, I take a backup of my database and load it into my local copy and play with it that way.

    Liked by 1 person

    Reply

  31. I remember reading your post when you explained how you compile a list of people who commented during the week. I thought about using a similar approach. I have a development environment similar to what I was working with before I retired. I’m sure I could write a program to handle all of the logic and reduce this ll to a button press. The extraction and filtering is similar to things I did while working, to gather data for one of our systems. That would limit me to a particular machine. When this covid confusion subsides, I do hope to travel again. Then again, we’ll have to see how bored I get next winter ;-)

    Like

    Reply

Add your thoughts or join the discussion. One relevant link is OK, more require moderation. Markdown is supported.

Fill in your details below or click an icon to log in:

Gravatar
WordPress.com Logo

You are commenting using your WordPress.com account. ( Log Out /  Change )

Google photo

You are commenting using your Google account. ( Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. ( Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. ( Log Out /  Change )

Cancel

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.