I want to thank all the people who weighed in on my post yesterday with comments about the Thursday Doors Recap page I am building. I appreciate the time you took to comment. I appreciate your encouragement and most of all, I appreciate the fact that so many of you were concerned about my time. I threatened promised to explain how I am preparing the summary. Of course, there’s a little bit of a backstory.
Note: if you find any typos, The Editor may have fallen asleep while reading this. If it affects you the same way, there will be no hard feelings if you skip to the gallery.
Since taking over the management of Thursday Doors, I have been concerned that people looking for the other doors posts have to wade through a lot of comments. That concern is compounded by the fact that people can contribute to Thursday Doors until noon on Saturday, but I have a regularly scheduled feature on Saturdays.
I was uncomfortable saying, “you can contribute on Saturday morning, but no one is going to see your link unless they start poking around my blog.”
Back in January, I made a list of participants and the URL of their TD post. I liked the list, but it took too long to prepare. I might have the time in the dead of winter, but when it gets warm, I’m not going to want to be assembling this list link by link. I have a subscription to LinkyTools, but my experience with that service is that some people are uncomfortable registering with “one more thing” in order to participate. Posting a link, or a pingback to No Facilities works, people are used to the procedure, why change to provide a feature people might not even want.
I started looking at software solutions for extracting links from a webpage – there are many – most were overkill and left me with one person’s idea of a perfect list. Then I stumbled on a post on Medium by Phil Gorman. He had a little bit of JavaScript (programming language) code that extracts links. In addition, he explained how it worked and where it could be modified. I liked that. I have worked with JavaScript before – it’s not my happy place – but I’m retired, I’m still capable of learning and I enjoy programming. On the other hand, I needed to know that a) this would work, and b) people would like it.
If you consider a blog page, with its comments and sidebar widgets, it might not surprise you that there are over a thousand links on that page. I modified the JavaScript to extract the links from only the comments section of my blog. That sounds good, but between links to avatars, blog posts, Like buttons, Reply buttons, Edit (because it’s my blog) buttons and the link to the comment itself, there are still a few hundred links and only two or three dozen matter. Of course, I can filter those links.
Since I am in the proof-of-concept stage, I decided to move the raw list into Excel. I heard several of you groan. I will spare you the details, but these are the steps Excel helps me execute:
- I delete all of my reply comments and the comments without links
- I delete the links to everything that’s not a comment
- I extract the Domain name (i.e. NoFacilities.com) from the URL of the blog post being added
- I collect the title from links added by Pingback
- I match the domain name from step three with a list of “friendly names” I created for those domains
- I create a hyperlink from the Pingback text or friendly name and the URL of the Doors post
- I copy the list of hyperlinks and paste it into the Block Editor (creating a table)
The steps listed above are mostly automatic. Ignoring for the moment the time it took me to do all this. It takes about 10 minutes from the time I extract the links to the time I paste the table into WordPress. I can live with that. If I move more of the process to the JavaScript program, I can probably reduce that to five minutes.
I am going to leave this technical kludge in place for a few weeks. One think about running the process in Excel, is I get to see the things that send it off the rails and work out a way to tweak the process steps to accommodate those things.
On Thursday, I will have a survey with a few questions for Thursday Doors participants, the results of which should help me improve this list and the process. I’ll also have some doors from downtown Waterbury, Connecticut. Today’s gallery is just the usual suspects.
Hi Dan – I noted … but loved the photos more … Arnold looks like he’s got tummy ache?! Must not disturb Mimi or MUmu – let them be! Those spikes are interesting to see … not common for us … all the best through the cold days – Hilary
LikeLiked by 1 person
Thanks Hilary. I understand this wasn’t an exciting read. Some people might like it but…
LikeLiked by 1 person
I think Arnold is just hungry 🙂
LikeLike
Rumbly tummy … and probably … good luck with the replies … cheers H
LikeLiked by 1 person
I have to say Thursday Doors has definitely found the right father. That’s dedication above and beyond.
And Arnold? I love him!
LikeLiked by 1 person
My wife loves that little guy.
Winter is a good time to do stuff like this.
LikeLiked by 1 person
I loved the geeky discussion, Dan, though your technical skills far exceed mine. I spend most of my working life as an analyst. When I retired as a full-time government guy and came back as a contractor, I started working with servers and scripts and such, which scare me, because I suddenly had the power to really mess the data up if I was not careful. I remember telling a colleague that I thought I was turning into a geek and he looked at me, smiled, and replied, “Turning into…?” I love the long shadows, as always, and the view of the flag. Thanks for the intro to Arnold–he looks like a cool squirrel.
LikeLiked by 1 person
I started our as a systems analyst, but I kept getting dragged into programming. I do enjoy it, as long as it’s not an accounting system. It’s fine to let your inner geek out no and then, Mike. As for having the power to really mess the data up, yeah, I’ve done that – cleaning up was a challenge, but we got through it. This is the first good picture I’ve gotten of Arnold. Most stand up with their butt on the ground. This guy looks like one tough hombre.
LikeLiked by 1 person
I know you know I flew right past the geeky stuff Dan! Lol.
Love the shadow shots of you and Maddie. Smokey looks fit as a fiddle. And now enters Arnold who is too cute for words. He has the “begging” pose down pat. That little guy couldn’t have found a better home.
I’m so impressed by that weed! What a good role model for all of us.
“Don’t touch my foot” is still cracking me up. 🤗
Hope this is the start of a great week for you and the Editor.
Ginger
LikeLiked by 1 person
Thanks Ginger. I knew a lot of people would fly over the geeky stuff. It’s OK. But I didn’t want people to worry about the time it takes me to prepare the list – a few hours of geeky stuff and then it’s easy peasy.
The Editor spied (and named) Arnold early in 2020, but it’s been hard to get a god picture of him. He’s just so dang cute. I think he knows it, and his “feed me” act always gets results.
That weed is my fav. I didn’t pull the snow away from the shed because I didn’t want to hurt it. That’s spring spirit if I’ve ever seen it.
I hope you have a great week, too.
LikeLike
Reminds me of a few projects from my earlier work life when ‘screen scraping’ was a thing. I understood your post and salute you for the research and the results. It is a kind thing you are doing to make spreading the word about Thursday Doors more streamlined. Always love your pics. Spring is just around the corner and it will be a great time to capture all of Mother Nature’s changes.
LikeLiked by 1 person
Thanks Maggie. We have a longer run to the corner, but spring is coming. March is usually the point where, even if we get snow, it’s gone in a few days. I remember screen scrapping exchange rates out of the Wall Street Journal online, before they finally relented and agreed to email us the table. The information is all there, it’s just a matter of finding and formatting.
LikeLiked by 1 person
I hope March is kind to you.
LikeLiked by 1 person
Sounds like you’ve found a solution that works. I skipped to the photos and enjoyed the journey, as I always do. Have a fantastic day, Dan.
LikeLiked by 1 person
Thanks Gwen. I think there’s room to improve this solution, but I’m happy with it the way it is.
LikeLike
I always love the usual suspects! You can never go wrong there.
LikeLiked by 1 person
Those guys know how to play to the audience.
LikeLiked by 1 person
I really did try to understand the geeky stuff. I will happily be part of the survey on Thursday. Loved the photos, especially sunsets, the flag, and of course Maddie. Have Smokey and Sammy welcomed Arnold?
LikeLiked by 1 person
Arnold sometimes shows up alone, but he lives in the area and we will see him with the crowd. But when he stands up, he definitely stands out. When two or more squirrels come together, they often fight over the first peanut they see, and then a blue jay swoops in and takes it. I’ve been known to say, “that’s what happens when you aren’t willing to let someone go first.” They don’t seem to be learning.
LikeLiked by 1 person
You’re right, they fight and the blue jay wins the peanut. Arnold has a beautiful white tummy. Yes, he must stand out in the crowd, and the lady squirrels must swoon.
LikeLiked by 1 person
Hahaha – it’s all about desire.
LikeLiked by 1 person
Yes! 🙂
LikeLiked by 1 person
I think it’s great that you’re working out a way to post a recap of the weekly contributions to Thursday Doors. I look forward to receiving it in my inbox.
Arnold is adorable! He looks very humble with that arm posture. ADORABLE!!
LikeLiked by 1 person
I’m glad you like the recap, Deborah. Arnold really his adorable. He just cracks us up when he comes up to beg. How could you say no?
LikeLiked by 1 person
😀
LikeLiked by 1 person
I skipped the geeky stuff – but I like Arnold may have a future as a linebacker.
LikeLike
Arnold does have that linebacker look, Frank.
LikeLike
I used to create tables and active links to files on maps for GIS so applaud your efforts. As I said before I would not be likely to click on the links to see other posts unless the table included the thumbnail link of the person who commented plus a link to their door post. As it is now I can scroll through and pick which ones with links that I want to check out at a glance and at the same time see who was posting. Making tables creates a whole lot of extra work for you. For your sake, I still think it is best to leave it alone. It is easy to see who provided a link and for me to skip by comments that don’t include links. I never have time to visit them all anyway. I fail to see how putting the links in a table without seeing who posted the link can be an improvement. I am basing this comment on the table you recently posted.
One way to streamline links in comments would be to have a rule that after one day comments, though appreciated, but not containing links to door postings will be removed after a response to eliminate clutter. This way only comments with links will be visible on Thursday Doors. This also entails extra work so that is not an ideal solution to the clutter either. Just a thought.
LikeLiked by 1 person
The idea of the list seems to be popular. As best as I can. I am forming the link from the person’s pingback or from a friendly name derived from the URL. It isn’t just the URL to the blog post, in many cases it’s the blog title or the author’s name that shows in the list. I’m going to give participants an option to make that name a bit more informative, but that will be something for people to opt into.
I wouldn’t want to close comments to people not providing a link because that would include the people who are just visiting my blog and commenting on my post.
This is one of those things that, if enough people like it, I think it’s worth what has become a minimal effort on my part. If people prefer to read through the comments, they will remain just as they are – this is in addition to that and I designed it with the hope of promoting the TD participants.
LikeLike
Great. As long as you have fun with it. I love Thursday doors even though I don’t post much lately.
LikeLiked by 1 person
Developing the list has been fun, since I don’t spend much time working in that area since I retired. The doors posts are always fun to write and read.
LikeLike
I love Arnold and the furry kids way more than Monday morning talk of Excel spreadsheets. Good luck with whatever you are doing and have fun being the Thursday door Grandmaster.
Have a wonderful Monday, Dan!
LikeLiked by 1 person
Thanks Mary. When I wrote the part about groaning, I almost said “I can hear Mary groaning” but I didn’t want to pick on you. Excel is one of the hardest things for me to use because it always takes me back to work.
I hope you have a great week.
LikeLike
On the positive side, I would say a post about Excel is still better than a post about algebra or liver. 😏
LikeLiked by 1 person
Hahaha – good to know there’s a hierarchy 😏
LikeLiked by 1 person
It sounds like you have things under control, Captain. Beam me up, Scotty. Just point me to the right ‘door’ for entry when all set up. 🤭🤦♀️😉
LikeLiked by 1 person
When it’s all set up, everything will be the same. On Saturday night or Sunday, I’ll post that the recap is ready – that’s it.
Kirk out.
LikeLike
Sounds stellar!
LikeLiked by 1 person
I wish you well with your Thursday Doors upgrade. It sounds doable, but not especially easy. As are most things when it comes to ye olde computer.
LikeLiked by 1 person
Thanks Ally. So far, the work has been fun, and the weekly process is very easy.
LikeLiked by 1 person
Arnold is adorable!! Smokey has competition in the ‘cuteness’ category. Oh, you geeky types…. A neighbor and I were talking this weekend cause she is retiring after this week. First thing on her to-do list is get her husband to clean out his mess. She said he has a box full of old laptops, routers, roller-ball mice, etc., that just need throwing out. I told my husband and could almost see his brain dancing with glee as he thought of all the ways he could ‘swap out parts’….which drives me nuts! Sounds like you have a great plan in store for doors, though! Go forth and have fun!
LikeLiked by 1 person
Haha – I tossed old laptops, desktops, and all the assorted bits last year. Including a laptop running Windows 2000! I’m sorry I didn’t check with you first.
Arnold is a little cutie. My wife loved him. I’ll stick to the deployment plan. No more talk about Excel.
LikeLiked by 1 person
Dare I repeat that Arnold is a cutie? I’d be out there with a bag of peanuts for sure. Appreciate you sharing the NanoSpikes. Appreciate your efforts on improving Thursday Doors. I did read the geeky stuff; still not totally convinced the list is necessary (I enjoy reading thru the comments …) but it also sounds like you’re able to make the computer do most of the work … and as long as it stays fun for you …
LikeLiked by 1 person
You can get on board the Arnold fan wagon. My wife is enjoying her buddy getting a better reception than my geek talk.
The list isn’t everyone’s cup of tea, but if it works for some and it’s easy to do, I’ll keep up with it.
The nano spikes are great on days like this. The dog wants to go for a walk, but there are patches of ice everywhere. The snow melts, has nowhere to go and then freezes.
Thanks.
LikeLiked by 1 person
I’ve always found Excel to be useful — and in unexpected ways. That’s where I keep a “character matrix” for each of my stories. I include a lot more than just the characters. You went to a lot of work, Dan. I’m sure the results pay off.
Tee-hee, I love “Arnold” and that foot closeup. Thanks for the photos. Stay safe and well. Hugs on the wing!
LikeLiked by 1 person
I remember you talking about how you use Excel for a character matrix. That’s a really good idea. It is a great tool. I have a better tool, a programming environment in which I could make this a dirt simple drill, but I can only run that on one laptop. If I keep this with a little JavaScript and a little Excel, I can do it anywhere, on any device I happen to have. In many ways, I learned that that kind of convenience is better than a slightly faster program.
Arnold is winning over hearts today.
Hugs!
LikeLiked by 1 person
Dan I admire your tenacity and keeping the hinges working on Thursday Doors. I hope WP does not do anything to throw a monkey wrench into your code. When you started describing all of the places links show up in a page all I could do was nod in commiseration. Will your routine for harvesting the links be effected by ‘slow’ internet ? Not that we ever run into that. The WP happiness engineers have had all sorts of problems with their lists over time. The most recent being some sort of truncation. I use the personal lists in the wp reader to follow blogs. There were a couple days this past week that the reader pulled back about 20 blogs and then just stopped like someone had truncated the size of the table. Like I said I hope the happiness engineers don’t do anything different to surprise you. If it were Saturday I would pick up the tab on the next round. Cheers.
LikeLiked by 1 person
I don’t think speed will affect the process, John, except for the fact that the whole page needs to be loaded. Once it’s loaded, the JavaScript runs in the browser’s console screen, and it’s pretty fast. The only thing WP can do to mess it up is change the nature of their static links (Reply, edit, that sort of thing). I wouldn’t put it past them to do that, so this will never be a set-it-and-forget-it kind of operation. I need to review the results.
LikeLiked by 1 person
You do what you geekily do, Dan, and I’ll live with the results and, I imagine, be happy. As long as TD continues, I’ll be good. But I understand what you mean about going through comments. Even if a person likes to read the comments, which i often do, it takes a lot of time and they might like to more easily access the links the way you’re showing. At any rate, I enjoyed the photos and enjoy sharing with you that our high today is supposed to be 77. Yikes! Hitting the floor running now.
janet
LikeLiked by 1 person
77? Wow. We’ve had 7, 17 and 27, but the possibility that the 7 will be first in a 2-digit number…unlikely for a while.
Working with the code and Excel was a fun project. I’m making a short list of things to change. Once I have enough of those, I’ll spend some more time with it. I’m not changing the way the comments work, but I hope to give people options. I think the list is a great way to catch up, especially with the people who paste later in the process.
LikeLiked by 1 person
My favorite geeky thing you’ve done is the link to TD the day before. Because I always schedule posts, it saves me from having to hustle to the host’s post (little rhyming there) and having to link.
LikeLiked by 1 person
I’m glad you like that, Janet. Again, once I set that up, it’s a trivial thing to create the link each week.
I will share a funny story. It took me two weeks before I realized that I just had to schedule my Thursday Doors post, I didn’t have to have it 100% ready to read on Tuesday night. All I really need is a title ;-)
LikeLiked by 1 person
I’m so glad this makes you happy, Dan. Oh, look a photo of Maddie. And there’s Smoky, MiMi and Arnold. Have a super day. 😁
LikeLiked by 1 person
Thanks John – it takes all kinds ;-)
LikeLiked by 1 person
😂
LikeLiked by 1 person
Aha, Arnold proves the reincarnation theory. . .I wonder who he was in his past life? We’ve had high winds gusting through the nights which doesn’t make for peaceful sleeping. Today it’s in high thirties which would be pleasant if not for wind!🌬
Sent from Yahoo Mail for iPad
LikeLiked by 1 person
I’m going to be careful with what I say about Arnold. My wife likes him.
LikeLiked by 1 person
Hey, Dan…the geeky stuff was way above my head. I love that your Editor names the squirrels that visit you. :)
LikeLiked by 1 person
She doesn’t name all of them, but this little guy is the only one who stands like a little body builder.
LikeLike
Okay, I think Arnold is officially my new favorite. :D Thanks for the geekery; it was very refreshing! I love the Linux version of Excel — very flexible and useful.
LikeLiked by 1 person
Smokey might be sad. Still, Arnold is quite a little hunk.
LikeLiked by 1 person
Arnold is cute, Dan. I have to start naming the creatures in my garden. I’m not a geek and you took 10 minutes to do what you did is amazing! I used to participate in Thursday Door long time ago. I ran out of doors!
LikeLiked by 1 person
Thanks Miriam. If you ever find a door, you are always welcome to drop in.
We only name the interesting ones. This guy is pretty cute, and he’s the only one that stands like that.
LikeLiked by 1 person
I surely will do that, Dan! It would be more fun for me to drop by when I find interesting doors.
LikeLiked by 1 person
I like the idea of a kludge for a few weeks. It gives you time to examine cases and build a better logic. Of course, it is your time, and you need to decide how much of it you want to spend.
My problem with such a process is the following, A year later a new special case will come along. By them I’ll have forgotten what I did (perhaps even the language I wrote in) and I will be too lazy to go through it again. I hope you can organize it better for yourself.
LikeLiked by 1 person
I am smiling at your future scenario, it is possible. I will try to keep track of the little things that run this off into a field. I’ve made notes about why I did what I did, and I may leave it as a kludge.
LikeLiked by 1 person
I did read your geeky stuff but my brain didn’t keep up. Cute squirrel.
LikeLiked by 1 person
Thanks.
LikeLike
Your photos are quite clear and crisp looking. Especially Arnold. I love that you care about the little weed poking out of the snow.
LikeLiked by 1 person
The weed might be a silly thing, but it made me smile to see it poking through the snow.
LikeLiked by 1 person
I get it. :)
LikeLiked by 1 person
According to what Firefox has sent me, their browser doesn’t recognize java. They don’t use it. Is this going to be a problem with your Thursday Doors project?
And just to let you know, the temperature reached 61 today (Tuesday). :P
LikeLiked by 1 person
I only need to run the JavaScrip program on my end. It runs well in the Firefox console, Once the list is prepared, it is formatted by WordPress as a standard blog page. I have viewed it in Firefox, Chrome, Safari and Microsoft Edge.
61? Wow – we got up to 39 and it felt like spring.
LikeLiked by 1 person
I think that sounded very geeky indeed. Kludge is a new word for me, thank you. I do think I need spike shoe things and also, I love Arnold – he is precious.
LikeLiked by 1 person
Arnold is fast becoming a crowd favorite 😊 Kluge is a good word.
LikeLiked by 1 person
Sounds like you have it down to a science. My approach is to have a local copy of WordPress running under MAMP (a prepackaged LAMP stack for Mac). Each week, I take a backup of my database and load it into my local copy and play with it that way.
LikeLiked by 1 person
I remember reading your post when you explained how you compile a list of people who commented during the week. I thought about using a similar approach. I have a development environment similar to what I was working with before I retired. I’m sure I could write a program to handle all of the logic and reduce this ll to a button press. The extraction and filtering is similar to things I did while working, to gather data for one of our systems. That would limit me to a particular machine. When this covid confusion subsides, I do hope to travel again. Then again, we’ll have to see how bored I get next winter ;-)
LikeLike